fix(compaction): symmetric heuristic stats pair and authoritative display (#2052)

[yangjj-iso]

Summary

Focused re-do of the #2052 compaction-stats fix. The earlier PR #2409 bundled this together with the #2328 skill-selector work; #2328 has since landed on main independently (d4dc771b0), so this PR carries only the #2052 change, rebased onto the latest main.

Compaction reported a token-reduction ratio whose two sides used different estimators: tokensBefore was a provider-authoritative count while tokensAfter was a char/4 heuristic, and the protected-tail estimate was reverse-engineered by subtracting one from the other. That mix could overshoot the real count and render a negative percentReduction, and the headline "Compacted from N tokens" line could disagree with the reported stats.

Changes

• computeWholeContextStats — one shared widening/ratio path used by the planned, fresh, and extension results. The protected tail is always part of the tokensBefore baseline and is added to tokensAfter only when it survives, so a kept tail cancels out of the ratio while a dropped tail is correctly counted as reduction. Both sides come from the same estimator.
• getKeptTailTokenEstimate fallback changed from tokensBefore - region.tokenEstimate to 0, with a new exported setKeptTailTokenEstimate so directly-built preparations set the tail estimate the way prepareCompactionBoundary does, instead of inferring an incommensurable mixed-unit value.
• displayTokensBefore — an authoritative tokensBefore is carried through the verbatim result/details and rendered in the boundary message, keeping the provider-aware display separate from the internally self-consistent heuristic stats. Resumed entries fall back to stats.tokensBefore when absent.
• Updated the compaction-small-region-fresh fixture to inject its oversized protected tail via setKeptTailTokenEstimate (the removed subtraction fallback no longer synthesizes it).
• Added compaction-stats-symmetry regression tests covering the symmetric ratio, the kept/dropped tail cases, and the authoritative display.

Testing

• npm run check — biome + tsc + coding-agent tsgo + shrinkwrap: green
• All compaction unit + integration suites (22 files, 148 tests): green

Closes #2052.

---

^{Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.}

Greptile Summary

This change separates the authoritative compaction display count from symmetric heuristic statistics, persists the display value, and adds coverage for resumed compaction boundaries and interactive rendering.

One P2 repository-convention issue remains: four new test imports use .ts source suffixes where the project requires .js ESM specifiers.

Confidence Score: 4/5

The functional compaction changes appear focused and covered by new regression tests, but the added test should conform to the project’s ESM source-specifier convention before merge.

There is one independent non-security P2 finding and no P0 or P1 findings, which maps to a score of 4.

Files Needing Attention: test/unit/compaction-stats-symmetry.test.ts lines 23-26 should use .js ESM source specifiers.

Prompt To Fix All With AI

### Issue 1
test/unit/compaction-stats-symmetry.test.ts:23-26
**Use ESM source specifiers**

These four new imports use `.ts` suffixes instead of the repository-required `.js` ESM specifiers. This makes the test inconsistent with the supported module-resolution convention and less portable across compilation and execution modes.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

_{Reviews (1): Last reviewed commit: "fix(compaction): symmetric heuristic sta..." | Re-trigger Greptile}

Greptile also left 1 inline comment on this PR.

Context used:

• Context used - AGENTS.md (source)

[greptile-apps]

Use ESM source specifiers

These four new imports use .ts suffixes instead of the repository-required .js ESM specifiers. This makes the test inconsistent with the supported module-resolution convention and less portable across compilation and execution modes.

Context Used: AGENTS.md (source)

Prompt To Fix With AI

This is a comment left during a code review.
Path: test/unit/compaction-stats-symmetry.test.ts
Line: 23-26

Comment:
**Use ESM source specifiers**

These four new imports use `.ts` suffixes instead of the repository-required `.js` ESM specifiers. This makes the test inconsistent with the supported module-resolution convention and less portable across compilation and execution modes.

**Context Used:** AGENTS.md ([source](https://github.com/bastani-inc/atomic/blob/main/AGENTS.md))

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

[flora131]

Thanks for putting this focused revision together. We’re starting a formal merge-readiness code review now, including current-main compatibility and the relevant checks. We’ll follow up here shortly with the review result and any required changes.

[flora131]

Thanks again for the focused #2052 work. We completed the merge-readiness review of this PR and confirmed that the symmetric estimator itself is sound.

While comparing it with #2407, we realized the branch choice is not as clear as we first thought. #2449 is the newer clean, focused PR, but #2407 is also focused on #2052 and already contains two useful fixes that are missing here: the required .js test import specifiers and an optional fallback for the internal compaction-boundary display.

Both branches still need the protected-tail preparation fix, documentation updates, and a rebase onto current main. Before we continue with detailed review feedback or close either PR, could you let us know which branch you want us to treat as the canonical implementation for #2052? We’ll keep both open until you confirm. Thank you.

[yangjj-iso]

Thanks for the careful comparison. Please treat #2407 as the canonical implementation for #2052.

It contains the same symmetric-estimator direction as #2449, plus the follow-up fixes for the dropped-tail tokensBefore baseline, the required .js ESM test specifiers, and the optional boundary-display fallback. Since both branches still need the protected-tail preparation work, documentation updates, and a rebase onto current main, I think continuing from #2407 gives us the more complete base and avoids reintroducing those fixes manually.

I’ll rebase #2407 onto current main, complete the remaining protected-tail/documentation work, and rerun the relevant checks. Once that is reflected, #2449 can be closed as superseded. Thank you.