Correct the dcast comment about which CBS buckets duplicate - #774
Open
lbm364dl wants to merge 1 commit into
Open
Correct the dcast comment about which CBS buckets duplicate#774lbm364dl wants to merge 1 commit into
lbm364dl wants to merge 1 commit into
Conversation
`.select_best_source()`'s justification for `fun.aggregate` recorded a measurement from #429: 31,642 duplicated (key, source) combinations at full range, in buckets 206 and 999, with a dump of bucket 999 in 2010 holding four distinct territories. Neither bucket duplicates any more: - 999 resolves all 62 `fabio_code == 999` areas to the one aggregate polity `ROW-1850-2025`, and since the un-fold became the default they are not folded into the bucket at all. - 206 stopped when #563 removed `polity_name` from `.aggregate_to_polities()`'s grouping keys, so a bucket folding several live territories now emits exactly one row per key. The issue measured 206 at 60fc661, which predates that fix. The comment is the recorded evidence for why `sum` is the right aggregate, and a reader reasoning about the Rest-of-World bucket from it was being told the duplication is still there — which cost a full re-measure in #555. Narrowed to history plus the two reasons it ended, and restated why `fun.aggregate` must stay: dcast's fallback is applied to every cell, so any future duplicate silently turns the whole table into row counts. Two tests pin what the comment now claims instead of a stale measurement: a duplicated (key, source) pair must come back summed (fails with `fun.aggregate` removed — the wheat cell becomes 2 and the unduplicated maize cell becomes 1), and 206 must be the only `polity_area_code` that folds more than one live polity in any year, derived from `polity_area_crosswalk` alone with no pins or network. Closes #557 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What was wrong
.select_best_source()inR/build_cbs.Rjustifies itsfun.aggregateargument with a measurement from #429 — 31,642 duplicated
(key, source)combinations at full range "in areas 206 and 999", plus a dump of bucket 999 in
2010 for wheat holding four distinct territories. That is the recorded evidence
for why
sumis the right aggregate, and it is also what a reader uses toreason about the Rest-of-World bucket. Neither bucket duplicates any more:
fabio_code == 999resolve to the single aggregatepolity
ROW-1850-2025, so even with the fold applied the bucket emits onerow; and since the un-fold became the default (
.unfold_rest_of_world(),"all") the members are not folded into it at all.polity_namefrom.aggregate_to_polities()'sgrouping keys (
b2526238), so a bucket folding several live territories nowemits exactly one row per key. The issue measured 206 at
60fc6610, which isnot a descendant of that fix — verified with
git merge-base --is-ancestor— so the 206 half of the claim was already stale when filedtoo, not only the 999 half.
Measured on this branch, off
polity_area_crosswalkalone:What changed
Comment only, no behaviour change. The measurement is kept but marked as
history, with the two reasons it ended, and the argument for
fun.aggregaterestated so it no longer depends on a live duplicate:
dcast()'s fallback islength()applied to every cell, so any future duplicate anywhere — abucket, a re-mapped item code, one key reported in two units, since
key_colsexcludes
unit— would silently turn the whole table into row counts.fun.aggregateis untouched.Two tests pin the property instead of a measurement:
test_build_cbs.R— a duplicated(key, source)pair comes back summed,and an unduplicated key in the same table keeps its tonnes.
test_polity_folds.R— 206 is the onlypolity_area_codefolding more thanone live polity in any year, and every
fabio_code == 999area resolves toROW-1850-2025. Crosswalk-derived: no pins, no network.How it was verified
A comment fix cannot have a test that fails before it, so both tests are shown
to be sensitive by mutation instead.
fun.aggregatedeleted from thedcast()call:(the wheat cell becomes the count 2 instead of 3,347,000, and the unduplicated
maize cell becomes 1 instead of 500 — the global-fallback bug itself)
Crosswalk assertion, with one ROW member given back a live national polity
inside bucket 999:
Restored, both files green:
air format .run;lintr::lint()clean on all three changed files (the repo.lintrkeepscommented_code_linteron, which the first draft of both testcomments tripped). No roxygen, no exports, no NSE symbols, so no
document()/_pkgdown.yml/globalVariables()change. NoNEWS.mdentry:no user-visible behaviour changes.
Classification: mechanical — documentation accuracy plus two assertions; no
number moves.
Closes #557.
🤖 Generated with Claude Code