fixtures(google): SPARKLINE conformance rows - #772
Merged
Conversation
115 rows exported from the Google Sheets conformance fixtures, evaluated
against live Sheets across eight probe batches. Every batch after the first
was triggered by a specific finding that could not be settled by argument.
114 of them go to google.tsv. The 115th,
`=MIN(SPARKLINE({1,2,3}),{})` → `#REF!`, is a known engine divergence and
goes to bugs.tsv, this repo's home for GS-captured rows the engine does not
yet match: MIN has no empty-array rule at all, so it answers 0 — for
`=MIN({})` too, with no sparkline involved. `=MAX({})` is `#REF!`, so MAX's
counterpart row passes. The gap predates sparklines and is unrelated to
them; fixing it from here would move MIN for inputs that have nothing to do
with this work. The row is recorded verbatim, so it flips to passing the
moment MIN is fixed and can then move to google.tsv.
The second batch supplied the control the first was missing:
`=SPARKLINE({1,2,3})=SPARKLINE({1,2,3})` → TRUE looked like evidence that a
sparkline's value identity is its parsed spec, but
`=SPARKLINE({1,2,3})=SPARKLINE({9,9,9})` → TRUE disproves it — every
sparkline compares equal to every other. COUNTUNIQUE (2 for different, 1
for identical) is what actually requires the spec to be retained. Two row
descriptions in that batch ("identity: different data is not equal",
"...different charttype is not equal") were written before the run and read
backwards against their own recorded TRUE; the recorded values, not the
descriptions, are the ground truth.
The third and fourth batches settled the currency/conversion split —
DOLLAR and FIXED reject a sparkline while the whole TO_* family answers ""
— plus the single-argument aggregates and the blanket numeric rejection
(ROUND, ABS, INT, unary minus and percent).
The fifth and sixth batches probe a sparkline arriving through a RANGE,
from a cell holding `=SPARKLINE(...)` (`Data!K1`, with 5 in `Data!K2`) —
how a real workbook delivers one. `=PRODUCT(Data!K1:K1)` is 0, so an
aggregate answers the same whichever way the sparkline arrived; the
two-cell rows cannot settle that alone, since 1 × 5 = 5 either way.
`=TYPE(Data!K1)` is 128: reading a cell is not a coercion point, despite
the empty display.
The seventh batch closes the aggregate family: every aggregate answers 0
when a sparkline is the only thing in scope — MAX, MIN, MAXA and MINA
included — with AVERAGE's #DIV/0! the single exception. MINA matching MAXA
was a coin-flip until probed: it had no implementation arm at all.
The eighth batch backs the one direct form that was still being inferred
from its range form (`=MIN(SPARKLINE({1,2,3}))` → 0) and finds a boundary:
an explicitly empty array argument outranks the sparkline skip
(`=MAX(SPARKLINE({1,2,3}),{})` → #REF!) while a text-only array does not
(`=MAX(SPARKLINE({1,2,3}),{"a"})` → 0).
Coverage: each charttype, an omitted charttype, colour options, ymin/ymax,
text inside the data, all-negative data, a genuine blank cell inside a real
source range, `bar` given three values, an unknown option key, an unknown
charttype value, a malformed options shape, the value-model probes, identity
and ordering against sparklines and scalars, the EQ/NE/GTE alias functions,
the three coercion seams, the TO_* family, presence (COUNTA, ISBLANK),
COUNTUNIQUE, and the aggregates direct, through a range, and against empty
and text-only arrays.
69 of google.tsv's 114 SPARKLINE rows are enforced by the conformance
runner. Of the other 45, 29 record an empty expected value (a rendered
chart has no text projection) and 16 read authored input cells the
standalone runner has no workbook to supply — it skips both, and now says
so per file (core#767). crates/core/tests/sparkline.rs asserts those cases
directly, the range rows against a seeded resolver.
One row is deliberately named as a parser artefact: `=SPARKLINE({1,,3})` →
`#ERROR!` is Sheets rejecting the array-literal syntax before SPARKLINE
runs, not a blank-cell rule. The `Data!H`/`Data!I` rows are the trustworthy
blank-cell probe and its all-present control.
Refs #766
Contributor
Test Coverage by Category
✓ = 100% passing · ⚠ = known deviation · The ~79,424 total counts formula evaluations (each conformance row and each property case = 1). GitHub Checks reports 3,732 Rust test functions: 2,897 unit + 159 property functions (shown as cases above) + 676 conformance/integration. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
closes #766
Summary
The Google Sheets conformance rows for
SPARKLINE, whose implementation merged in #770. Fixtures-only — theCheck fixture / code separationCI job rejects any PR touching both these TSVs and code, which is why this is separate.115 rows, every value observed in live Google Sheets. 114 to
google.tsv; 1 tobugs.tsv— see below.Why this took 115 rows and not 26
Ground truth started at 26 and grew across eight probe batches. Not one was planned. Each came from a review finding where two readings were equally plausible and only Sheets could adjudicate — and in several cases the plausible reading was wrong:
=SPARKLINE({1,2,3})=SPARKLINE({1,2,3})→TRUEwas read as identity is the parsed spec. The missing control,=SPARKLINE({1,2,3})=SPARKLINE({9,9,9}), is alsoTRUE. Every sparkline equals every other. That reversed the value model after it was already built.DOLLAR→#VALUE!butTO_DOLLARS→"". Near-identical names, opposite answers.TO_PERCENT→""while its structurally identical siblings were assumed to error. Three of the five-memberTO_*family had the wrong answer.PRODUCTwith a lone sparkline is0, not the multiplicative identity — invisible with two arguments, which is why it passed for a while.MINAmatchingMAXAat0was a coin-flip until asked.The one row in
bugs.tsv=MIN(SPARKLINE({1,2,3}),{})→#REF!does not pass. This engine returns0, becausemin_fnhas no empty-array rule whilemax_fndoes.That divergence is pre-existing and has nothing to do with sparklines —
=MIN({})is0onmainwith no sparkline involved. Fixing it would moveMINfor unrelated inputs, so it is recorded rather than papered over, in the file this repo documents for captured values the engine does not yet match. It is non-blocking, counted, and flips to passing the moment #771 lands — at which point it moves to its category TSV.The alternative was quietly dropping an inconvenient row. That is how ground truth gets curated into agreement with the implementation.
How to test
google.tsvis a blocking gate. The tally now prints what is actually enforced versus skipped, which #770 added after the previous coverage claim turned out to be overstated:The 29 have no recorded value because a rendered sparkline has no text projection; the 16 read authored cells the standalone runner cannot resolve. All 45 are asserted directly in
crates/core/tests/sparkline.rs(47 tests) instead. That gap is #767.Provenance
Every row came from the Google Sheets conformance pipeline — evaluated in a real spreadsheet, transcribed verbatim, never computed by this engine. The 14 pre-existing ARRAYFORMULA rows are untouched.
One methodological note worth recording: in an early batch, 9 probe formulas were written without a leading
=, so Sheets stored them as literal text and returned plausible-looking garbage while the run reported success. It was caught by reading the values rather than trusting the exit status — and one of the nine was the control row that reversed the value model. Every later batch asserted the=prefix before running.Test plan
cargo test -p truecalc-core --test conformance— 20 passed, 0 failedgoogle_conformancegreen as a blocking gatemainRelated
MIN's empty-array divergence, which thebugs.tsvrow records🤖 Generated with Claude Code
Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.