fix(doctor): name the check groups dropped when config load fails - #10
Draft
benw5483 wants to merge 2 commits into
Draft
fix(doctor): name the check groups dropped when config load fails#10benw5483 wants to merge 2 commits into
benw5483 wants to merge 2 commits into
Conversation
benw5483
force-pushed
the
doctor-surface-skipped-checks
branch
2 times, most recently
from
August 5, 2026 21:37
566349c to
ba7265c
Compare
`gc doctor` gates seven groups of checks on the city config loading cleanly. When that load fails, every pack, pack-source, config-validation, rig, data, session and Dolt-ops check disappears without a trace. The run prints a shorter summary that names the config error itself but says nothing about which families of checks were dropped, so a summary reading "36 passed" can describe a factory that was barely inspected. Measured on a city whose config does not load, before and after: before 41 checks, no skipped count, none of the dropped groups named after 48 checks, "7 check groups skipped", every group named Worth stating plainly, because it narrows what this actually fixes: the exit code was already non-zero in that state. The pre-existing expanded-config-load check fails blocking whenever the config cannot be resolved. What was missing is the inventory, not the exit status. Two halves: Surface it. Each config-dependent group now registers a blocking SkippedCheck naming what did not run, the summary states how many groups were skipped, and the run ends with an explicit line saying the factory was not fully inspected. The underlying config error is carried as a detail rather than repeated on all seven lines, where it buried the group names it was meant to explain. Remove the cause. Pack scripts run with their working directory set to the pack dir, which for an imported pack lives inside gc's global import cache. gc exported GC_CITY_PATH but never FACTORY_ROOT, so a script resolving its state dir against FACTORY_ROOT fell back to the working directory and wrote into the cache clone. That left the clone dirty, and the next gc invocation refused the dirty cache, failed config load, and dropped every pack check, which is the loop that produced the numbers above. PackRuntimeEnv now exports FACTORY_ROOT. Both halves carry positive and negative controls: the skip must appear when config load fails and must stay absent when it succeeds. Each was verified by injecting the defect and confirming the test goes red. Generated by the operator's software factory. City: factory-main · Agent: local-core.builder-3 On behalf of: @benw5483 Co-Authored-By: <operator-factory-bot> <factory-bot@operator-domain.invalid>
benw5483
force-pushed
the
doctor-surface-skipped-checks
branch
from
August 5, 2026 21:47
ba7265c to
1b23bb7
Compare
Two review follow-ups on the skipped-check-inventory change. cmd/gc/testdata/doctor.txtar — the unit tests prove the skips are registered, but nothing proved they reach stdout. A later change to runDoctor that filtered or short-circuited output on the config-failure path would keep every unit test green while the operator saw nothing, which is the same silent-invisibility defect this change exists to close. The fixture already built a broken city and ran the real binary, so the three assertions ride along on it. Each assertion was proven non-vacuous by injection, one at a time, against an untouched upstream tree: with only 'pack-doctor-checks' added the fixture goes red naming that string, and the same holds for 'check groups skipped' and 'not fully inspected'. All three pass together on this branch. internal/doctor/doctor.go — the doc comment on Report.Skipped asserted that skipped groups are always counted in Failed and BlockingFailed. That is false once a --fix run repairs the cause mid-run: SkippedCheck downgrades to StatusWarning and tally counts it in Warned, which TestReportCountsSkippedGroupsWhenWarning already asserts. The field is projected to JSON, so a consumer trusting the old wording could gate CI on skipped > 0 implying blocking_failed > 0. The comment now states the blocking case, names the --fix exception, and warns off that inference. Generated by the operator's software factory. City: factory-main · Agent: local-core.builder-3 On behalf of: @benw5483 Co-Authored-By: <operator-factory-bot> <factory-bot@operator-domain.invalid>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
gc doctorregisters seven groups of checks only when the city config loads cleanly, and when that load fails all seven vanish from the run with nothing said about them. The summary gets shorter. It also reads healthier, because the checks that were failing left along with the rest, so you're looking at a shrunken run that looks like a better one.Two halves, matching the two ways this can be fixed:
Surface it. Each config-dependent group now registers a blocking
SkippedChecknaming what didn't run, the summary line gains anN check groups skippedpart, and the run ends with a sentence saying the factory was not fully inspected. The config error itself moves into per-result details. Repeating it on all seven lines just buried the group names it was there to explain.Remove the cause. Pack check scripts run with their working directory set to the pack dir, which for an imported pack lives inside gc's global import cache. gc exported
GC_CITY_PATHbut neverFACTORY_ROOT, so a script resolving its state directory againstFACTORY_ROOTfell back to the working directory and wrote into the cache clone. Now the clone is dirty. The next gc invocation refuses the dirty cache, fails config load, and drops every pack check, which closes the loop back to the first half of this PR.PackRuntimeEnvandPackRuntimeEnvMapnow exportFACTORY_ROOT.Measured
I built both binaries and ran each against the same city, one whose config doesn't load. The environment was isolated (
env -i, scratchHOME) so that neither run could reach a real city and quietly measure the wrong thing:skippedin--json7The seven named on the after run:
pack-doctor-checks,pack-source-checks,config-validation-checks,rig-checks,data-checks,session-checks,dolt-ops-checks.One thing this does not change, and I'd rather say it than have you find it: the exit code was already non-zero here. The pre-existing
expanded-config-loadcheck fails blocking whenever the config can't be resolved, so both binaries exit 1 on this city. What was missing is the inventory of what went unrun, not the exit status.Controls
Both directions are pinned. I verified each by injecting the defect rather than by reading the test and trusting it:
TestBuildDoctorChecksRegistersSkipsWhenConfigFails: on an unloadable config every group in the list must register, and the result must be error + blocking, carry the config error in its details, and namegc import installin its fix hint.TestBuildDoctorChecksNoSkipsWhenConfigLoads: on a healthy config no skip result may appear. Without it, every clean run would claim the factory went uninspected, andgc doctorwould never exit 0 again.TestConfigDependentCheckGroupsAreDeclaredpins the group list against an independent copy of the names.cmd/gc/testdata/doctor.txtaralready built a broken city and ran the real binary, so the three newstdoutassertions ride along on it:pack-doctor-checks,check groups skipped,not fully inspected. The unit tests prove the skips register; these prove they reach the operator. A later change torunDoctorthat filtered or short-circuited output on this path would keep every unit test green while the operator saw nothing, which is the same silent-invisibility defect the PR exists to close. Run it withGC_FAST_UNIT=0 go test ./cmd/gc/ -run 'TestTutorial01/doctor$'.Each of those three was injected separately against an untouched upstream tree rather than checked as a set. With only
pack-doctor-checksadded the fixture goes red naming that string, and the same holds one at a time forcheck groups skippedandnot fully inspected. All three pass together here. Injecting them as a group would have proved only that the first one bites, since the fixture stops at its first failed assertion.That last one exists because of something the injection pass caught. Both of the first two tests iterate
configDependentCheckGroups, so the list was simultaneously the subject and the oracle: I injected a deletion of thepack-source-checksentry and everything stayed green, because the tests just looped over a shorter list and checked less. That is the standard weakness of a list that grades itself. With the guard in place the same injection goes red.Be clear about what the guard does not cover, though, since it's the more interesting half. It catches a group being removed or renamed. It cannot catch a new
cfgErr == nilgated block being added tobuildDoctorCheckswith no matching entry here — that block would still be dropped silently, which is exactly how thepack-source-checksgap arose in the first place. Catching that needs a tripwire over the gated blocks themselves, and I left it out on purpose: every future PR that adds a gated block would have to update it, and imposing that on other contributors is a call for the maintainers rather than for this PR. The maintenance contract is written on the list, and I'd rather flag the gap than quietly widen scope.The skip re-reads the config when it runs instead of trusting the value captured at build time, so a cause that
--fixrepairs mid-run downgrades to a warning rather than reporting a failure that isn't there any more.TestSkippedCheckWarnsWhenCauseRepairedMidRunpins that direction andTestSkippedCheckStillBlocksWhenCausePersistspins the other.One deliberate call on the resource census
The first push tripped
TestRepositoryLedgerMatchesCensusAndDocumentation: my new test file usedt.Setenv("GC_DOLT", "skip")twice, which pushed thecmd/gc+untaggedenvironment debt from 4318 to 4320 calls and 202 to 203 files.Raising the two baselines in
test/test-resources.tomlwould have made it green. I didn't, because that ledger's stated invariant is that existing debt cannot grow, and quietly bumping it is the failure mode the ratchet exists to catch. Thet.Setenvturned out to be unnecessary anyway:buildDoctorChecksnever readsGC_DOLT, onlyrunDoctordoes, and it passes the result down throughbuildDoctorChecksOpts, which both tests already set. So I dropped both calls, the ledger is untouched, and there's a comment at the call site recording why the tests don't mutate the environment. Verified by running the two tests withGC_DOLT=onand with it unset; both pass either way.Test plan
go test ./internal/doctor/... ./internal/citylayout/...— passing locallygo test ./cmd/gc/...— full package, no narrow-rungofmt -lclean andgo vetclean on all nine touched filesgo test ./internal/testpolicy/resourcecensus/— the ledger shard, which the first push trippedconfigDependentCheckGroupscoverage check I asked for below: confirmed, no gap today. Review walked all eightcfgErr == nilgated register blocks inbuildDoctorChecksand mapped them onto the seven entries, withdolt-ops-checkscovering both the main-block Dolt registrations and the separately-gated postgres-auth one. One block reads like a ninth and is not: theif cfg != nilfurther down only selectsdoctorCfg, and the two registers under it are unconditional, so the worktree checks correctly need no entry. The unenforced invariant stays a future-regression risk rather than a live gap, and the tripwire for it is deferred with a stated reason.Note for reviewers
The
pack-source-checksgroup covers the pack import cache check and the pack-source credential check. The latter arrived with the credentialed-pack-imports work and is config-gated, so it belongs in this inventory.