Skip to content

fix(terminology): consume verified cache bytes - #39

Open
TheHalfMoon wants to merge 1 commit into
mainfrom
fix/terminology-verified-cache-bytes
Open

fix(terminology): consume verified cache bytes#39
TheHalfMoon wants to merge 1 commit into
mainfrom
fix/terminology-verified-cache-bytes

Conversation

@TheHalfMoon

@TheHalfMoon TheHalfMoon commented Aug 26, 2026

Copy link
Copy Markdown
Owner

Scope

Fix issue #35 by binding terminology closure parsing to the exact bytes returned by the verified cache boundary.

Canonical base:

main: eeecb0bc03c7040bb18b70bce8b69d618384f783
tree: d5abe932f1436a9612f45bf130ba29aadbc5a133
AF-01 planning: CANONICAL

Candidate head:

e463eb0ca2cb04130abdc02574294d99eb9db291

Defect

TerminologyClosure::load previously called lockfile.verify_cache(cache) and then independently reopened each cache archive with fs::read. A local mutation between verification and the second read could cause terminology analysis to consume bytes whose digest was never verified.

Repair

  • remove verify-all then raw-reopen behavior;
  • read each package exactly through PackageCache::read_verified;
  • pass that returned Vec<u8> to manifest parsing, package inspection, and resource scanning;
  • keep all existing package identity, duplicate canonical, malformed canonical, and fail-closed semantics;
  • add a deterministic unit regression that injects the verified-reader boundary and proves one reader invocation per archive with no second filesystem input path.

Explicit non-goals

Qualification

Merge only from an exact head with mandatory CI and all path-applicable proof/oracle workflows terminal green, fresh Qodo and CodeRabbit review truth, and zero unresolved substantive findings.

Closes #35.


Summary by cubic

Fixes a cache verification gap in TerminologyClosure::load where packages were verified, then re-read from disk before parsing, leaving a window where mutated bytes could be consumed. Packages are now read exactly once through the verified cache reader, so terminology analysis always operates on digest-verified bytes.

Repair

  • Replaces the verify-then-reopen flow with a single PackageCache::read_verified call per archive.
  • Passes the verified bytes to manifest parsing, package inspection, and resource scanning without a second filesystem access.
  • Adds a regression test asserting one verified-reader invocation per package.
  • Preserves existing identity, duplicate canonical, malformed canonical, and fail-closed behavior.
  • Closes fix(terminology): consume the exact cache bytes that were verified #35.

Written for commit e463eb0. Summary will update on new commits.

Review in cubic

Summary by CodeRabbit

  • Bug Fixes
    • Improved package archive loading reliability by using verified cached archives.
    • Ensured each locked archive is accessed only once during terminology processing.

@qodo-code-review

Copy link
Copy Markdown

ⓘ Your Qodo trial ends soon. Ask your workspace admin to set up billing to keep reviews running after the trial. Manage billing

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your trial has ended. Reactivate Greptile to resume code reviews.

Copy link
Copy Markdown
Owner Author

@qodo review

Please review exact current head e463eb0ca2cb04130abdc02574294d99eb9db291. Focus on the verified-byte invariant, whether load_with_reader preserves existing terminology semantics and fail-closed behavior, whether the one-read regression is meaningful, and whether any unverified cache reopen remains in this terminology closure path. Do not carry findings or approval from other PRs.

Copy link
Copy Markdown
Owner Author

@coderabbitai review

Please perform a fresh review of exact current head e463eb0ca2cb04130abdc02574294d99eb9db291. Focus on cache TOCTOU closure, API/test design, potential semantic regressions, and whether terminology now consumes exactly the bytes whose digest was verified. No CodeRabbit PASS is assumed.

@qodo-code-review

Copy link
Copy Markdown

PR Summary by Qodo

Fix terminology parsing to consume verified cache bytes

🐞 Bug fix 🧪 Tests 🕐 10-20 Minutes

Grey Divider

AI Description

• Consume terminology archives directly from cache verification, eliminating the verify/read race.
• Reuse verified bytes for manifest validation, inspection, and resource scanning.
• Prove each locked archive crosses the verified-reader boundary exactly once.
Diagram

sequenceDiagram
    participant C as Terminology Caller
    participant T as Closure Loader
    participant P as Package Cache
    participant A as Archive Analysis
    participant I as Canonical Index
    C->>T: Load lock closure
    loop Locked packages
        T->>P: Read verified digest
        P-->>T: Verified archive bytes
        T->>A: Parse same bytes
        A-->>T: Manifest and resources
        T->>I: Insert canonicals
    end
    T-->>C: Terminology closure
Loading
High-Level Assessment

Using the existing PackageCache::read_verified boundary is the appropriate repair because downstream archive APIs already consume byte slices. Keeping bulk verification followed by raw filesystem reads would preserve the race, while streaming verification would require a broader archive-processing redesign without improving this focused fix.

Files changed (1) +66 / -7

Bug fix (1) +66 / -7
terminology_index.rsConsume verified archive bytes throughout terminology closure loading +66/-7

Consume verified archive bytes throughout terminology closure loading

• Replaces verify-all followed by raw filesystem reopening with one 'read_verified' call per locked package, then reuses those bytes for manifest checks, inspection, and resource scanning. Adds an injectable reader seam and a generated archive regression proving exactly one verified read per package.

crates/commandf-pkg/src/terminology_index.rs

@coderabbitai

coderabbitai Bot commented Aug 26, 2026

Copy link
Copy Markdown

Review Change Stack

Important

Review skipped

No new commits to review since the last review.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 0f073133-b6b9-4159-bc61-2d07181da97c

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 8b3b9d90-3ef5-4f4c-9397-0789931c23b0

📥 Commits

Reviewing files that changed from the base of the PR and between eeecb0b and e463eb0.

📒 Files selected for processing (1)
  • crates/commandf-pkg/src/terminology_index.rs

Included review availability: Your plan provides up to 10 included reviews per hour; 3 remain after this review.


📝 Walkthrough

Walkthrough

TerminologyClosure::load now reads package archives through PackageCache::read_verified using an injected reader. Tests use an in-memory archive and verify one request per locked archive.

Changes

Terminology verified loading

Layer / File(s) Summary
Verified reader loading
crates/commandf-pkg/src/terminology_index.rs
load delegates archive retrieval to load_with_reader and removes direct filesystem reads after cache verification.
Reader boundary regression
crates/commandf-pkg/src/terminology_index.rs
Tests provide a gzip-compressed manifest archive in memory, verify the locked digest is requested once, and cover absent value-set resolution.

Estimated code review effort: 2 (Simple) | ~10 minutes

Merge Risk: ⚪ Minimal · up to e463e

The change makes terminology processing consume the exact bytes returned by verified cache reads while preserving existing validation and fail-closed behavior; no actionable merge-blocking risk remains beyond normal checks and review.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 40.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 5 functions across 1 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed The changes satisfy issue #35 by routing terminology archive reads through PackageCache::read_verified, passing verified bytes to parsing and inspection, and adding a deterministic regression that ver…
Out of Scope Changes check ✅ Passed The changes are limited to the terminology cache-read repair and its regression coverage. No unrelated AF-01, CF-06, CF-10, or terminology redesign work is shown.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: terminology processing now consumes verified cache bytes.
Full details: Linked Issues check

Explanation

The changes satisfy issue #35 by routing terminology archive reads through PackageCache::read_verified, passing verified bytes to parsing and inspection, and adding a deterministic regression that verifies one reader request per archive. The summary also indicates that existing identity, duplicate, malformed canonical, and fail-closed behavior remains unchanged.

✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/terminology-verified-cache-bytes

Comment @coderabbitai help to get the list of available commands.

@qodo-code-review

Copy link
Copy Markdown

review

Please review exact current head e463eb0ca2cb04130abdc02574294d99eb9db291. Focus on the verified-byte invariant, whether load_with_reader preserves existing terminology semantics and fail-closed behavior, whether the one-read regression is meaningful, and whether any unverified cache reopen remains in this terminology closure path. Do not carry findings or approval from other PRs.

Review result

I found no correctness issues in the current head for the areas requested.

  • Verified-byte invariant: TerminologyClosure::load now supplies cache.read_verified to load_with_reader (crates/commandf-pkg/src/terminology_index.rs:30-35). For each locked package, the returned bytes are reused for manifest validation, inspection, resource scanning, and JSON parsing (:46-65); there is no subsequent archive read.
  • Terminology semantics and fail-closed behavior: The existing manifest identity check, inspection/scanning errors, duplicate filename rejection, canonical URL filtering, duplicate canonical detection, sorting, and resolution behavior remain intact (:48-116, :119-163). Errors from the verified cache reader propagate through TerminologyError::Package, so missing, malformed, or digest-mismatched cache objects still fail closed.
  • One-read regression test: closure_consumes_each_archive_from_verified_reader_once is meaningful: it would catch an implementation that invokes the supplied reader more than once for a package, and it verifies that the loaded closure remains usable (:248-275). It is intentionally a narrow call-count test; because the injected reader is a test double and the archive contains only a manifest, it does not independently prove hash verification or exercise terminology-resource indexing. Production verification is established by the load wiring to read_verified.
  • Unverified cache reopen: None remains in this terminology-closure path. The prior verify_cache plus direct root()/sha256/*.tgz read was removed; the only cache access from TerminologyClosure::load is read_verified.

No approval or findings are carried over from another PR.

@qodo-code-review

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0)

Grey Divider

Great, no issues found!

Qodo reviewed your code and found no material issues that require review

Grey Divider

Tip of the day
💡 Did you know, you can start a comment with 'qodo' or '@qodo' to chat about any finding

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Grey Divider

Qodo Logo

@coderabbitai

coderabbitai Bot commented Aug 26, 2026

Copy link
Copy Markdown
🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

requested='e463eb0ca2cb04130abdc02574294d99eb9db291'
echo '== checked-out HEAD =='
git rev-parse HEAD
echo '== requested commit =='
git rev-parse "$requested^{commit}"

echo '== diff summary at requested commit =='
git show --stat --oneline --decorate --no-renames "$requested"

echo '== changed source and test diff =='
git diff "${requested}^" "$requested" -- crates/commandf-pkg/src/terminology_index.rs

echo '== terminology index outline =='
ast-grep outline crates/commandf-pkg/src/terminology_index.rs --items all --view expanded

echo '== PackageCache declarations and verified-read call sites =='
rg -n -C 4 --glob '*.rs' \
  '(struct\s+PackageCache\b|impl\s+PackageCache\b|fn\s+read_verified\b|\bread_verified\s*\(|\bverify\s*\()' \
  crates

echo '== terminology closure call sites and archive/path reads =='
rg -n -C 5 --glob '*.rs' \
  '(TerminologyClosure|load_with_reader|TerminologyClosure::load|fs::read|read_to_end|File::open|archive_path|cache_path)' \
  crates/commandf-pkg

Length of output: 29459


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo '== PackageCache implementation =='
sed -n '1,180p' crates/commandf-pkg/src/cache.rs

echo '== TerminologyClosure implementation and tests =='
sed -n '1,310p' crates/commandf-pkg/src/terminology_index.rs

echo '== Terminology error definitions and conversions =='
rg -n -C 6 --glob '*.rs' \
  '(enum\s+TerminologyError\b|impl\s+From<PackageError>|PackageError|fn\s+read_manifest\b|fn\s+scan_terminology)' \
  crates/commandf-pkg/src

echo '== Exact cache verification followed by direct cache-object reads =='
rg -n -U -P --glob '*.rs' \
  '(?s)\.verify\(&?[A-Za-z0-9_.$\[\]]+\.sha256\)\?;.{0,900}(?:fs::read|File::open|archive_path|object_path|read_locked_archive)\s*\(' \
  crates

echo '== All terminology closure loading and lockfile cache verification uses =='
rg -n -C 3 --glob '*.rs' \
  '(TerminologyClosure::load\s*\(|\.verify_cache\s*\()' \
  crates

Length of output: 85082


@TheHalfMoon Fresh review completed for e463eb0ca2cb04130abdc02574294d99eb9db291.

I found no findings in the requested terminology closure scope.

TerminologyClosure::load now obtains each archive through PackageCache::read_verified. It passes the returned bytes to manifest parsing, package inspection, and resource scanning. This closes the prior verify-then-reopen TOCTOU gap in this path.

load_with_reader preserves the existing identity validation, terminology parsing, duplicate-resource handling, canonical validation, and fail-closed error propagation.

The regression uses a valid archive and asserts one reader invocation per locked archive. It guards against another archive acquisition inside the closure.

No direct cache-path reopen remains in crates/commandf-pkg/src/terminology_index.rs.

I also found existing verify-then-reopen patterns in separate CLI flows. They are outside this terminology closure path and outside this PR diff.

I completed a static source review. I could not execute repository tests in this sandbox.

⚠️ Action not completed

Already reviewed the last commit. Use @coderabbitai full review to rerun a review of the entire changeset.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(terminology): consume the exact cache bytes that were verified

1 participant