Commit the guides' tutorial data instead of fetching it at build time

[GondekNP]

The two data-driven guides declared what data they need, but the files were not in the repository.
They were preprocessed on every push from a zip fetched over the network — landing/test_preprocess.sh:19
does wget https://joshsim.org/guides/raw_tutorial_supplement.zip, from the site the build publishes to.

So the input to the build was a mutable URL. If that zip changes, the published guides change with
no commit and no diff. If it 404s, four jobs fail. This commits the data instead.

Why this is affordable

.jshdz is XZ-wrapped .jshd, and it compresses extraordinarily well:

file	.jshd	.jshdz
precipitation	0.71 MB	23 KB	32×
precipitationTulare	1.40 MB	6 KB	225×
temperatureTulare	1.40 MB	5 KB	296×
total	3.5 MB	34 KB

34 KB against a 64 MiB pack. The repo already commits four .jshd files under paper/, so
committing data is not a new policy here — only the compression is.

Why nothing else had to change

MultiFormatExternalGetter resolves a bare external name to <name>.jshdz before <name>.jshd, so
external precipitation finds the compressed file with no model edits and no jar changes. And
because the committed name already matches the external name, the rename the old pipeline needed
(precipitation_geotiff.jshd → precipitation.jshd) goes away.

The one constraint

.jshdz is JVM-only — the XZ decoder is not compiled through TeaVM (build.gradle:47). The
guides tell readers to upload the data into the browser editor, so the site must keep serving
.jshd. deployStatic now expands each committed file as it stages the guides, which also retires
the hardcoded three-file copy: the guide directory names the destination, so adding a guide with
data needs no change there.

The page says this out loud rather than leaving a reader puzzling over a filename they cannot find —
the Data row names the file as the repository stores it and notes that the download is the expanded
form.

CI simplification

job	before	after
testPreprocessing	runs preprocess, uploads artifact	runs preprocess — no longer decides what ships
testGuideExamples	downloads artifact	reads the repo
testAdditionalFeatures	downloads artifact	the download was dead — none of its four scripts read it
deployStatic	downloads artifact, hardcoded 3-file copy	expands from the repo, generic

testPreprocessing keeps running: exercising preprocess against netCDF and GeoTIFF is worth doing.
It just no longer decides what ships.

Also carried

The first commit declares the data: field on both guides, which was missing — the manifest recorded
that the models reach precipitationTulare/temperatureTulare/precipitation but not which files
provide them. The harvest now fails both when a runnable unit reads an external no declared file
provides, and when a declared file is not beside its model — a check the field could not have while
the data lived in an artifact. Conformance tests are exempt from the first; their fixtures are staged
by the test harness.

Verification

• xz -d returns the published bytes exactly — all three round-trip byte-identical to what the
  site serves today
• The deploy step was simulated locally: produces the same three paths with identical bytes
• Run time is unchanged: two_trees takes 24s from .jshdz against 26s from .jshd
• examples/test_guide_examples.sh passes against the committed data, from a clean tree with no
  hand-staged files
• 164 Python tests pass, ruff clean. Negative controls confirmed: a typo'd data name and a
  declared-but-absent file each fail the harvest with a file and line

Known gaps

• examples/test.sh also stages from landing/preprocessed_data/ and is in no workflow — the
  same shape as paper/management/test.sh. Left alone; nothing fed it before this change either.
• landing/test_preprocess.sh still wgets the raw zip. That is now only in the test path, not the
  publish path. Making it diff its output against the committed copy — so a preprocessing regression
  fails the build instead of quietly changing the guides — needs byte-reproducible preprocess
  output, which I have not verified.
• deployStatic is skipped on PRs, so its rewiring first runs when this reaches dev.

🤖 Generated with Claude Code