Repair ten functions, remove the recursion and residue, and compile src/ into the workbook

[ryanduguid]

Summary

This actions every finding from the workbook review: the functions that returned wrong answers on ordinary inputs, the calculation hot spots, the verification gap, the residue and determinism items, and the documentation that contradicted the code. All eight CI gates pass under Python 3.11 and 3.12, and the 216 tool tests (39 of them new) pass.

Functions that returned wrong answers

• oz.Depreciateλ no longer errors on a disposal before the end of life (EXPAND cannot shrink); the remaining book value is written off in the disposal month.
• oz.Amortiseλ builds its default timeline from every loan, and reads a text start date.
• oz.IsOccurrenceDateλ finds monthly, quarterly, semi-annual and annual items that start on the 29th to 31st in shorter months.
• oz.PeriodLabelλ's ISO week label carries the ISO year.
• oz.FinancialYearλ reads a text date; a blank cell still returns a blank.
• The two by-item schedulers score an item with no schedule rows as nought rather than #CALC!, and no longer recurse.
• oz.IsInListλ and oz.IsInListUλ search a row, column or grid, without wildcard semantics; their copied Name Manager comments now describe them.
• The Debt module returns errors instead of the help table.
• oz.CashRatioλ spills two columns again; oz.SumPeriodsλ drops a date before the first period instead of erroring the row.

Calculation that scales

• The Debt module solves each schedule with SCAN instead of one recursion level per period, and oz.InterestLRVλ is solved in closed form (reproduces 222.90 and every self-test value). With nothing recursing by name, Debt joins the AFE store and verify_afe.py requires all six modules.
• oz.Depreciateλ, oz.Amortiseλ (sub-monthly timelines) and oz.SumPeriodsλ aggregate months into periods with one matrix product. oz.Periodsλ, oz.CorkScrewReversalλ, oz.Movementλ, oz.LabelAmortiseλ and the four rolling functions are vectorised.

Tooling

• tools/compile_sources.py: renders src/ into the stored defined names (_xlfn./_xlws./_xlop./_xlpm. markers, SINGLE() for @, oz. qualification, Excel's case normalisation), proves each rendering through verify_sources.py's own comparison, refuses anything it cannot classify, rewrites only changed definitions, sets Name Manager comments from source headers and regenerates functions.csv. It compares literal text exactly (layout whitespace and the [0]! marker aside), and verify_sources.py now holds the same line; the one definition that had drifted inside a literal, oz.AboutEssentialsλ's About row, is recompiled.
• tools/verify_help_spills.py: a read-only check that models TRIM and TEXTSPLIT over each stored help literal and reports which cached help tables no longer match their definitions, without Excel. It never writes. On the tracked workbook it names exactly the ten helps this PR changed and matches Excel's cache on the other 33. RELEASING.md runs it before the native gates.
• tools/postbuild/remove_residue.py: drops the hidden FMTs sheet, the 38 custom properties, the stale custom-function declaration on the Excel Labs reference and the unused differential formats and named styles (references renumbered), and freezes the label columns on the six wide demonstration sheets. 211 parts become 169; 443,448 bytes become 430,473.
• sanitise_workbook.py pins the last editor (or removes it when the file names no creator), the modified stamp, window geometry, build stamp and revision pointer, so two saves of the same content agree.
• tools/generate_selftest_examples.py and tools/selftest_examples.ps1: 134 help assertions and 158 worked-example assertions derived from the help, dot-sourced by excel_selftest.ps1; the tool tests fail when the fragment is stale. Totals and tolerances are exact decimal sums of the printed digits, so the fragment regenerates identically under Python 3.11, 3.12 and 3.13.

Documentation

README walkthrough for oz.Amortiseλ, the implicit-rate conversion advice (README and lease help), the copied ratio help, the Debt help convention, the misspellings, and the run order in tools/postbuild/README.md.

Evidence and what remains

• Excel was not available where this was made. Every rewritten function was re-implemented in Python and run over its demonstration sheet's inputs; the cached cells it feeds came back unchanged (Periodsλ, both by-item schedulers, Movementλ, RollingSumλ, IsOccurrenceDateλ across 2,190 cells, SumPeriodsλ, LabelAmortiseλ), and the new depreciation aggregation matched the old one over 300 randomised timelines. No cached value was written by XML tooling: the workbook is rebuilt from main through the pipeline, and every cached value it carries is one Excel wrote.
• Ten help caches are stale by design until Excel refreshes them: the demonstration sheets for Periodsλ, ScheduleRatesByItemsλ, ScheduleValuesλ, ScheduleValuesByItemsλ, Depreciateλ, Movementλ, RollingSumλ, MinColsλ, IsOccurrenceDateλ and OverLapDaysλ show the previous help table. python tools/verify_help_spills.py ozzit.xlsx lists them; tools/refresh_cache.py in Excel, then tools/sanitise_workbook.py, refreshes them, and the manifest is realigned after that commit.
• The native gates still need to run on this candidate: tools/excel_selftest.ps1 and tools/verify_cache.py, and the workbook should be opened in Excel once, per the sanitiser's own advice. RELEASING.md now records 730 assertions (438 hand-written plus 292 generated); the generated figure is a static count, so please record the number the script prints.
• The workbook manifest is realigned after each workbook commit, as the manifest test requires. main (the RELEASING.md colon change) is merged into the branch.

🤖 Generated with Claude Code

https://claude.ai/code/session_01TntLJVhtvQFSQdoB2jBh9t

[qodo-code-review]

PR Summary by Qodo

Repair workbook functions and add deterministic source compilation

🐞 Bug fix ✨ Enhancement 🧪 Tests 📝 Documentation ⚙️ Configuration changes 🕐 40+ Minutes

AI Description

• Corrects date, scheduling, debt, depreciation, ratio, and list function edge cases.
• Replaces recursive and repeated calculations with scalable scans and matrix operations.
• Adds deterministic source compilation, workbook cleanup, help refresh, and comprehensive
 self-tests.

Diagram

graph TD
  SRC["Source Modules"] --> COMP["Source Compiler"] --> WB["Workbook Names"] --> POST["Postbuild Passes"] --> REL["Release Workbook"]
  SRC --> GEN["Test Generator"] --> TEST["Excel Self-test"]
  SRC --> AFE["AFE Store"]
  WB --> AFE

Loading

High-Level Assessment

The following are alternative approaches to this PR:

1. Excel COM-based build pipeline

• ➕ Uses Excel's native formula parser and recalculation engine
• ➕ Avoids independently modelling stored formula markers and help spills
• ➖ Requires Windows and a compatible Excel installation
• ➖ Introduces automation instability and machine-specific workbook state
• ➖ Cannot run consistently on existing CI runners

2. Continue manual workbook synchronization

• ➕ Avoids maintaining custom OOXML compilation logic
• ➕ Keeps Excel as the sole formula-authoring environment
• ➖ Requires duplicate edits across source, workbook, AFE, and index views
• ➖ Preserves the drift that caused stale comments and definitions
• ➖ Provides weaker determinism and reviewability

Recommendation: Keep the PR's source-first, deterministic OOXML approach. Its strict classification, round-trip verification, idempotency tests, and refusal of unknown states mitigate the main risk of a custom compiler while remaining CI-compatible; native Excel should remain the final arithmetic and cache-validation gate.

Files changed (28) +3302 / -390

Enhancement (2) +14 / -34

sync_afe_store.pySynchronize Debt into the AFE store +6/-13

Synchronize Debt into the AFE store

• Includes all six modules and every shipped function in AFE synchronization now that Debt no longer recurses by name.

tools/sync_afe_store.py

verify_afe.pyVerify all six modules and names in AFE +8/-21

Verify all six modules and names in AFE

• Removes the recursive-Debt exception and requires the AFE store to match every source module and shipped workbook name.

tools/verify_afe.py

Bug fix (6) +623 / -285

Dates.txtRepair date recurrence, labels, scheduling, and financial-year handling +75/-82

Repair date recurrence, labels, scheduling, and financial-year handling

• Handles short-month recurrences and ISO week years correctly, accepts text financial-year dates, vectorizes period calculations, and removes recursion from by-item schedulers. Empty item schedules now return zero instead of calculation errors.

src/Dates.txt

Debt.txtReplace recursive debt schedules with SCAN-based calculations +402/-14

Replace recursive debt schedules with SCAN-based calculations

• Rewrites all five Debt functions with structured help, explicit validation, and non-recursive schedule calculations. InterestLRVλ now uses a closed-form solution, and runtime errors are no longer replaced by help tables.

src/Debt.txt

Essentials.txtMake IsInListλ support exact grid searches +5/-3

Make IsInListλ support exact grid searches

• Flattens row, column, or grid inputs before exact matching, preventing wildcard interpretation of asterisks and question marks. Also corrects associated help text.

src/Essentials.txt

Financial.txtRepair and vectorize financial schedule functions +126/-176

Repair and vectorize financial schedule functions

• Fixes early depreciation disposal, multi-loan amortization timelines, and pre-period SumPeriodsλ dates. Replaces recursive or repeated calculations across amortization, movement, rolling, reversal, labeling, and aggregation functions with scans, direct array operations, and matrix products.

src/Financial.txt

Ratios.txtCorrect ratio help spills and argument documentation +10/-7

Correct ratio help spills and argument documentation

• Restores CashRatioλ's two-column help spill, validates BVPSλ's preferred-stock argument, and corrects misleading ratio descriptions, examples, and typographical errors.

src/Ratios.txt

Utilities.txtMake IsInListUλ support exact grid searches +5/-3

Make IsInListUλ support exact grid searches

• Uses exact XMATCH semantics over flattened list inputs and documents row, column, and grid support. Also fixes a help-text spelling error.

src/Utilities.txt

Tests (10) +1287 / -35

excel_selftest.ps1Load generated function help and example assertions +9/-3

Load generated function help and example assertions

• Dot-sources the generated assertion fragment and makes Debt help-row checks resilient to inserted documentation rows.

tools/excel_selftest.ps1

generate_selftest_examples.pyGenerate native Excel assertions from inline examples +304/-0

Generate native Excel assertions from inline examples

• Parses every function's help, filters examples requiring external inputs, and generates type-aware assertions for numbers, booleans, dates, labels, arrays, and spill shapes. It also adds a no-argument help check for every shipped function.

tools/generate_selftest_examples.py

selftest_examples.ps1Add generated coverage for all workbook functions +298/-0

Add generated coverage for all workbook functions

• Adds help checks for all 134 functions and assertions derived from 134 standalone worked examples, contributing 292 native Excel assertions.

tools/selftest_examples.ps1

test_idempotency.pyAdd new postbuild passes to idempotency coverage +2/-0

Add new postbuild passes to idempotency coverage

• Requires residue removal and help-spill refresh to be no-ops on the current workbook.

tools/tests/postbuild/test_idempotency.py

test_refresh_help_spills.pyTest help spill parsing, resizing, and idempotency +101/-0

Test help spill parsing, resizing, and idempotency

• Validates cached help tables against the TRIM/TEXTSPLIT model, checks spill growth and rewritten values, and rejects malformed multi-arrow rows.

tools/tests/postbuild/test_refresh_help_spills.py

test_remove_residue.pyTest workbook residue removal and reference repair +182/-0

Test workbook residue removal and reference repair

• Reintroduces each residue type into workbook copies and verifies exact cleanup, style and sheet-reference renumbering, freeze panes, idempotency, and malformed-state rejection.

tools/tests/postbuild/test_remove_residue.py

test_afe_store.pyRequire non-recursive Debt content in AFE +29/-32

Require non-recursive Debt content in AFE

• Updates AFE tests to require the Debt module and its five names, verifies Debt functions do not call themselves, and adjusts supported-encoding success expectations.

tools/tests/test_afe_store.py

test_compile_sources.pyTest source compiler rendering and workbook updates +178/-0

Test source compiler rendering and workbook updates

• Covers Excel markers, identifier classification, implicit intersection, comments, selective rewrites, index regeneration, unknown-name rejection, round-trip equivalence, and check-mode immutability.

tools/tests/test_compile_sources.py

test_sanitise_workbook.pyTest deterministic session-state sanitization +44/-0

Test deterministic session-state sanitization

• Simulates differing editor, timestamp, window, revision, and Excel-build metadata and verifies sanitization restores the tracked deterministic workbook bytes.

tools/tests/test_sanitise_workbook.py

test_selftest_examples.pyTest generated self-test parsing and assertions +140/-0

Test generated self-test parsing and assertions

• Ensures the committed fragment is current and ASCII, covers every function, excludes external examples, and correctly handles each supported result and help layout.

tools/tests/test_selftest_examples.py

Documentation (5) +187 / -27

CHANGELOG.mdDocument repaired formulas and deterministic build pipeline +130/-0

Document repaired formulas and deterministic build pipeline

• Adds a detailed unreleased entry covering correctness fixes, vectorized calculations, source compilation, generated self-tests, residue removal, and documentation corrections.

CHANGELOG.md

README.mdAlign workbook usage and verification guidance with the new pipeline +18/-14

Align workbook usage and verification guidance with the new pipeline

• Documents source-driven compilation, the actual Amortiseλ interface, annual-to-period lease-rate conversion, six-module AFE coverage, generated assertions, and deterministic sanitization.

README.md

RELEASING.mdRaise the native self-test acceptance baseline +1/-1

Raise the native self-test acceptance baseline

• Updates release evidence requirements from 438 to 730 assertions and distinguishes handwritten from generated checks.

RELEASING.md

functions.csvRefresh function signatures and corrected descriptions +9/-9

Refresh function signatures and corrected descriptions

• Updates generated Debt signatures and descriptions, corrects list and ratio documentation, and normalizes FinancialYearλ metadata.

functions.csv

README.mdDocument source compilation and new postbuild passes +29/-3

Document source compilation and new postbuild passes

• Adds the required run order and explains compilation, help-cache refresh, residue removal, AFE synchronization, generated assertions, and cache verification responsibilities.

tools/postbuild/README.md

Other (5) +1191 / -9

workbook-base.jsonRecord the rebuilt workbook artifact +4/-4

Record the rebuilt workbook artifact

• Updates the workbook blob, commit, checksum, and reduced artifact size after compilation and residue removal.

release/workbook-base.json

compile_sources.pyCompile readable source modules into workbook defined names +588/-0

Compile readable source modules into workbook defined names

• Adds a strict compiler for Excel markers, optional and local parameters, implicit intersection, library qualification, and identifier casing. It round-trip verifies every rendering, updates Name Manager comments and functions.csv, supports check mode, and rewrites only changed definitions.

tools/compile_sources.py

refresh_help_spills.pyRefresh cached help spills without Excel +227/-0

Refresh cached help spills without Excel

• Models Excel's TRIM and TEXTSPLIT behavior to rewrite changed help-table caches and resize their spill ranges. It rejects malformed help rows and leaves current caches byte-identical.

tools/postbuild/refresh_help_spills.py

remove_residue.pyRemove unused workbook parts and freeze wide-sheet labels +325/-0

Remove unused workbook parts and freeze wide-sheet labels

• Removes the hidden FMTs sheet, custom properties, stale custom-function metadata, and unused styles while renumbering references. It also freezes label columns on six wide demonstration sheets and enforces idempotent, anchored transformations.

tools/postbuild/remove_residue.py

sanitise_workbook.pyCanonicalize Excel session metadata +47/-5

Canonicalize Excel session metadata

• Pins the last editor, modified timestamp, and workbook window while removing revision pointers and Excel build metadata. This makes equivalent saves deterministic across users and machines.

tools/sanitise_workbook.py

[qodo-code-review]

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📜 Skill insights (0)

1. Literal whitespace skips compilation ✓ Resolved 🐞 Bug ≡ Correctness

Description

tight() removes whitespace inside every string literal, so semantically different formulas such as
"a b" and "ab" compare equal. apply() then leaves the old workbook definition unchanged and
--check incorrectly reports that it matches src/.

Code

tools/compile_sources.py[R388-395]

+def tight(formula: str) -> str:
+    """The stored form with no whitespace and no workbook-scope markers.
+
+    Runs of whitespace inside help literals are what TRIM() removes when the help is
+    read, and [0]! is how Excel marks a name it resolved in this workbook; neither
+    is a difference worth rewriting a definition for.
+    """
+    return re.sub(r"\s+", "", formula).replace("[0]!", "")

Evidence

The comparison helper globally applies re.sub(r"\s+", "", formula), and apply() uses that result
both to decide whether to skip a definition and whether to retain its existing body. Formula sources
contain non-help string literals used as calculation output, including the spacer literals in
Amortiseλ, proving that quoted whitespace is not exclusively disposable help formatting;
verify_sources.canonical() has the same blind spot and therefore cannot catch the resulting
mismatch.

tools/compile_sources.py[388-395]
tools/compile_sources.py[475-490]
src/Financial.txt[780-788]
tools/verify_sources.py[148-162]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description

The compiler's equality normalization removes whitespace from the entire formula, including quoted string literals. This can cause a meaningful source change to be skipped while the workbook retains the previous formula.

## Issue Context

Only syntactic whitespace outside quoted literals should be ignored. Whitespace inside help literals may be normalized separately when it is provably under the help table's `TRIM`, but arbitrary formula literals must remain exact.

## Fix Focus Areas

- tools/compile_sources.py[388-395]
- tools/compile_sources.py[466-490]
- tools/tests/test_compile_sources.py[111-134]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

2. Saver metadata remains unsanitized ✓ Resolved 🐞 Bug ⛨ Security

Description

The new sanitization pass only rewrites lastModifiedBy and modified when dc:creator exists. A
workbook lacking that optional creator element therefore retains the previous saver identity and
timestamp, defeating the stated privacy and deterministic-build guarantees.

Code

tools/sanitise_workbook.py[R190-193]

+    core = parts["docProps/core.xml"].decode("utf-8")
+    creator = re.search(r"<dc:creator>([^<]*)</dc:creator>", core)
+    if creator:
+        new_core = re.sub(

Evidence

Both substitutions are nested beneath if creator, even though they target separate XML elements.
The module documentation explicitly says the pass removes or pins the saver account and save
timestamp, so leaving both untouched in the creator-absent case violates its new contract.

tools/sanitise_workbook.py[16-25]
tools/sanitise_workbook.py[188-205]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description

Core-property sanitization is incorrectly conditional on finding a `dc:creator` element. Workbooks without that property retain `cp:lastModifiedBy` and `dcterms:modified` values.

## Issue Context

The modified timestamp should always be pinned when present. `lastModifiedBy` should be removed or replaced independently, using the creator only when one is available.

## Fix Focus Areas

- tools/sanitise_workbook.py[188-205]
- tools/tests/test_sanitise_workbook.py[1-44]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

3. ozzit.xlsx missing from changes ✓ Resolved 📘 Rule violation ≡ Correctness

Description

The PR modifies bound publication views including src/Dates.txt and functions.csv, but does not
include the corresponding ozzit.xlsx workbook. These downstream changes therefore cannot be tied
to a workbook update in the same change set.

Code

src/Dates.txt[7]

+        "About:                 →Ozzit compliant LAMBDAs dealing with dates. Suggested module name: oz¶" &

Evidence

The diff changes a published source definition and regenerated function index, while the change-set
inventory contains no ozzit.xlsx patch. Updating only release/workbook-base.json with a workbook
hash does not include the workbook itself.

Rule 3013985: Keep bound publication view files consistent with ozzit.xlsx
src/Dates.txt[7-7]
functions.csv[8-8]
release/workbook-base.json[2-7]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Bound publication files were modified without including the corresponding `ozzit.xlsx` update in this change set.

## Issue Context
PR Compliance ID 3013985 requires `ozzit.xlsx` to be changed whenever direct `src/*.txt`, AFE store, or `functions.csv` publication views change.

## Fix Focus Areas
- src/Dates.txt[7-7]
- functions.csv[8-8]
- release/workbook-base.json[2-7]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

4. Help caches bypass Excel ✓ Resolved 📘 Rule violation § Compliance

Description

refresh_help_spills.py computes formula results in Python and writes them directly into worksheet
cache <v> nodes without native Excel recalculation. The documentation then treats these
XML-generated values as current despite lacking verifiable Excel recalculation evidence.

Code

tools/postbuild/refresh_help_spills.py[155]

+                    kept.append((column, f"{head}{formula}<v>{text}</v></c>"))

Evidence

The new tool explicitly states that it performs no COM, no recalculation, then emits <v> values
for the formula anchor and spill cells. The README claims this makes the cached-value gate have
nothing to report, implementing the freshness guarantee through XML manipulation rather than native
Excel recalculation or evidence from it.

Rule 3014032: Do not modify cached formula results without verifiable Excel recalculation evidence
Rule 3014039: Use native Excel recalculation gates instead of XML tooling for formula or cache updates
tools/postbuild/refresh_help_spills.py[12-25]
tools/postbuild/refresh_help_spills.py[155-160]
tools/postbuild/README.md[35-41]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The postbuild tool directly fabricates cached worksheet formula results through OOXML edits rather than obtaining them from Excel's recalculation engine.

## Issue Context
The help anchors are formula cells such as `=oz.Nameλ()`. Their cached spill values must be produced or verified through native Excel recalculation, not treated as valid solely because a Python implementation models `TRIM` and `TEXTSPLIT`.

## Fix Focus Areas
- tools/postbuild/refresh_help_spills.py[12-25]
- tools/postbuild/refresh_help_spills.py[122-160]
- tools/postbuild/README.md[35-41]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

Context sources

✅ Compliance rules (platform): 13 rules

Review mode: ⚖️ Balanced

Tip of the day

💡 Did you know, you can turn on the rule miner and Qodo learns your standards from review history

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗