Skip to content

feat(sandbox): add quarantined patch validation - #65

Open
seonghobae wants to merge 108 commits into
mainfrom
feat/quarantined-patch-validation
Open

feat(sandbox): add quarantined patch validation#65
seonghobae wants to merge 108 commits into
mainfrom
feat/quarantined-patch-validation

Conversation

@seonghobae

@seonghobae seonghobae commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds a credential-free, allowlisted patch-validation boundary for exact-head review evidence. Untrusted source, patch content, repository scripts, Git status and exact-tree output, archive metadata, extracted filesystem objects, and validator output remain outside GitHub App, reviewer-model, NVIDIA NIM, Cloudflare, OIDC, publication, deployment, and Docker-socket credentials.

Buyer-visible gap addressed

Noema could quarantine source for graph inspection but lacked a bounded contract for validating a proposed patch against an authenticated exact Git revision without crossing a credential boundary. This PR supplies that evidence plane while preserving separation among check runs, commit statuses, review evidence, model judgement, protected-branch approval, build provenance, release acceptance, and deployment authority.

Implemented boundary

  • exact repository/base/head/patch-SHA-256/profile request and result binding;
  • enum-only validation profiles with no caller-provided shell command;
  • immutable digest-pinned validator image reference and --pull=never;
  • strict UTF-8 patch preflight for binary payloads, symlink/gitlink modes, noncanonical aliases, traversal, absolute/control-character/raw-backslash paths, malformed quoting, duplicate/oversized file sets, independent file/rename/copy metadata families, exact path-role binding, hunk counts, and governance-sensitive paths;
  • descriptor-safe, no-follow patch, Git control-file, and result reads with inode/device stability and distinct byte limits;
  • configuration-isolated exact Git HEAD and clean-worktree verification covering tracked, staged, untracked, and ignored drift;
  • constant-space dirty-worktree detection that reads at most one status byte, terminates the Git child immediately on drift, and accepts an empty stream only after a zero exit within the shared deadline;
  • incrementally streamed git ls-tree -r -l -z --full-tree preflight before archive allocation, retaining at most one bounded partial record and permitting only canonical 100644/100755 blobs with valid object identities;
  • exact-tree ceilings of 20,000 records, 4 KiB per path, bounded per-record metadata, 16 MiB aggregate tree metadata, 64 MiB per blob, and 512 MiB aggregate blob bytes, with early child termination on the first violation;
  • configuration-isolated exact-commit archive materialization into an owner-only source snapshot with export-ignore and export-subst neutralized;
  • pre-extraction allowlisting of normalized regular files and populated directories only, followed by exact lstat path/type/size equality before Docker receives the source;
  • regular-directory and linked-worktree .git handling with symlink/special-object refusal and type-compatible empty metadata boundaries;
  • private owner-only staged patch copy so the original caller pathname never enters Docker mount grammar;
  • no network, read-only root, non-root UID/GID, capability drop, no-new-privileges, seccomp, isolated IPC, no Docker socket, and bounded CPU/memory/PID/ulimit/tmpfs/wall-time resources;
  • one pre-created bounded /output/result.json evidence file with stdout/stderr discarded;
  • unknown-field refusal, duration/excerpt/reason-code bounds, PASSED→zero-exit consistency, and exact command re-binding;
  • beginner-readable public documentation, CHANGELOG.md, and APA 7th standards doctoring.

Addressed review and CI findings

  • Removed unused imports reported by GitHub Code Quality.
  • Removed every temporary self-modifying contents: write repair workflow; no .github/workflows/repair-* file remains.
  • Rejected raw backslashes and noncanonical path aliases before normalization.
  • Bound ---, +++, rename, and copy paths to the active primary diff identity while permitting complete canonical metadata families.
  • Enforced newline-marker placement and exact hunk line counts.
  • Enforced the 16 KiB result ceiling during descriptor reads.
  • Added prearchive exact-tree validation before git archive storage allocation.
  • Addressed the exact-head maintainer boundedness finding test-first: status output now stops after one byte; exact-tree output is parsed incrementally with explicit path, record, member, metadata-byte, per-file, and aggregate-file ceilings; child processes are terminated on the first violated bound.
  • Added RED and regression coverage for oversized exact-tree output, dirty status output, partial records, invalid UTF-8, stream timeouts, nonzero exits, terminate-to-kill escalation, archive failure, and every defensive streaming branch.
  • Restored 100% production statement and branch coverage and 100% public docstring coverage.
  • Updated public and doctoring documentation to describe authenticated Git metadata, bounded streamed Git evidence, archive/extraction equality, and the single result-file evidence channel.

Exact-head status

Current head: 9e03b17d0bdda75809f394aaafb0b44cf1267a51

  • ci: completed successfully
  • Security Scan: completed successfully
  • reviewer-ci: completed successfully — 331 passed; 100% production statement coverage; 100% production branch coverage; 100% public docstring coverage
  • distroless CodeGraph sandbox verification: Cosign success, Trivy 0 MEDIUM/HIGH/CRITICAL findings, and real no-network smoke success
  • review threads: all resolved or outdated; thread resolution is not approval evidence
  • prior maintainer streaming finding: addressed at this exact head; the maintainer comment itself was explicitly not an approval
  • eligible independent APPROVE: not submitted
  • CodeRabbit: exact-head review request is bound to base 9adb784902cae19a150146a8fadd6a0491f67814 and head 9e03b17d0bdda75809f394aaafb0b44cf1267a51, but CodeRabbit reported the review limit was reached and the review did not start. Rate limiting is not review success.
  • PR state: ready for review; merge remains prohibited until the current automated-review and independent-approval gates complete

Scope boundary

This PR adds a tested library and evidence contract. It does not yet build, publish, sign, scan, attest, or activate the dedicated patch-validator image in the reviewer decision flow. Those remain explicit follow-on gates. No version bump or release is claimed.

Merge policy

Do not merge unless the live exact head remains current, every required check and security gate succeeds, current human and automated findings are addressed, every addressed thread is resolved, CodeRabbit/current automated review is complete where repository policy requires it, and an eligible independent reviewer submits APPROVE. Queued, pending, or rate-limited checks and reviews are not success, and no protection may be bypassed or weakened.

Related: #9

@coderabbitai

coderabbitai Bot commented Aug 5, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@seonghobae, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 1 minute

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 5ab3d615-d27a-4264-bfc2-b9a095509b07

📥 Commits

Reviewing files that changed from the base of the PR and between 9adb784 and 89b4e5c.

📒 Files selected for processing (23)
  • CHANGELOG.md
  • docs/doctoring/quarantined-patch-validation.md
  • docs/quarantined-patch-validation.md
  • reviewer/noema_reviewer/__init__.py
  • reviewer/noema_reviewer/patch_validation.py
  • reviewer/tests/test_patch_validation.py
  • reviewer/tests/test_patch_validation_archive_boundaries.py
  • reviewer/tests/test_patch_validation_canonical_paths.py
  • reviewer/tests/test_patch_validation_coverage_edges.py
  • reviewer/tests/test_patch_validation_exact_tree_and_output.py
  • reviewer/tests/test_patch_validation_git_control_descriptor_edge.py
  • reviewer/tests/test_patch_validation_git_control_isolation.py
  • reviewer/tests/test_patch_validation_git_metadata_mask.py
  • reviewer/tests/test_patch_validation_hardening.py
  • reviewer/tests/test_patch_validation_mode_boundaries.py
  • reviewer/tests/test_patch_validation_path_consistency.py
  • reviewer/tests/test_patch_validation_prearchive_and_result_channel.py
  • reviewer/tests/test_patch_validation_provenance_and_hunk_edges.py
  • reviewer/tests/test_patch_validation_security_boundaries.py
  • reviewer/tests/test_patch_validation_source_integrity.py
  • reviewer/tests/test_patch_validation_streaming_edges.py
  • reviewer/tests/test_patch_validation_streaming_final_edges.py
  • reviewer/tests/test_patch_validation_streaming_git_output.py

Comment @coderabbitai help to get the list of available commands.

Comment thread reviewer/noema_reviewer/patch_validation.py Fixed
@seonghobae
seonghobae marked this pull request as ready for review August 5, 2026 07:16

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

Please review exact current head c83fb483ede9c6378b7e89c3b92a49d767140119. Treat prior draft-skip status and findings anchored to older heads as stale. Focus on the credential boundary, raw diff-path parsing, descriptor-safe reads, Docker isolation, exact request/result binding, bounded evidence, tests, public documentation, and standards claims.

@coderabbitai

coderabbitai Bot commented Aug 5, 2026

Copy link
Copy Markdown

@seonghobae I will review pull request #65 at exact head c83fb483ede9c6378b7e89c3b92a49d767140119.

I will treat findings on earlier heads as stale. I will focus on the specified security boundaries, validation logic, evidence limits, tests, documentation, and standards claims.

⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copy link
Copy Markdown
Contributor Author

@opencode-agent @cwl-noema-review

Independent exact-head review requested for c83fb483ede9c6378b7e89c3b92a49d767140119. Please submit a GitHub APPROVE only if this exact head satisfies repository policy, security boundaries, tests, documentation, and required checks. GitHub's formal reviewer-request endpoint rejected these identities because one or both are not repository collaborators; this comment does not substitute for an eligible independent approval.

Comment thread .github/workflows/repair-pr65-reviewer-ci.yml Fixed

Copy link
Copy Markdown
Contributor Author

@opencode-agent

Implement the valid exact-head CI finding on current head b1b00fa2dc97335b5a7192e13f34fb46565bbd0a; refuse to write if the head moves.

reviewer-ci remains red because test_runner_rejects_non_exact_source_worktree_before_docker[tracked|untracked] proves that a Git checkout with tracked or untracked drift still reaches Docker. Add a fail-closed, non-shell, bounded git status --porcelain=v1 --untracked-files=all preflight using TRUSTED_GIT_EXECUTABLE and the existing scrubbed Git environment before sandbox launch. A dirty or unverifiable Git worktree must raise before Docker. Preserve the separately documented behavior for authenticated non-Git source snapshots. Keep the tests test-first, maintain 100% production statement/branch and public-docstring coverage, update authoritative docs/CHANGELOG only where the contract changes, then verify all checks against the new exact head. Do not alter the existing review-agent key contract; use the development agent's NVIDIA NIM credential path only.

@seonghobae
seonghobae marked this pull request as ready for review August 5, 2026 17:29

@seonghobae seonghobae left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exact-head security review for c604f929df89c387b1e28d6892247b2ce810e15d: one boundedness gap remains blocking despite green coverage.

_verify_exact_tree_limits() invokes git ls-tree -r -l -z --full-tree with stdout=subprocess.PIPE through subprocess.run, so the entire hostile exact-tree listing is accumulated in memory before MAX_SOURCE_ARCHIVE_MEMBERS, aggregate byte, and path validation runs. A timeout is not a byte bound; a repository with far more than 20,000 entries can emit a large result quickly and exhaust runner memory before the fail-closed parser executes. _verify_source_head() has the same pattern for git status --porcelain=v2 --untracked-files=all --ignored=matching, even though any first output byte is already sufficient to reject a dirty worktree.

Please implement streaming, configuration-isolated reads with explicit byte/record ceilings and early process termination; bound individual path bytes and aggregate metadata bytes before archive creation; add RED regressions proving the reader never accumulates beyond the configured ceiling for oversized exact trees and dirty-worktree output. Preserve exact-head refusal and do not add repair workflows or write-capable Actions. @opencode-agent

This is not an approval and the PR must remain unmerged until this finding, current automated review, and eligible independent approval are complete.

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

Please review exact current head c604f929df89c387b1e28d6892247b2ce810e15d. The PR is ready for review. Treat findings anchored to earlier heads as stale or superseded. Focus especially on fail-closed bounded subprocess output, exact-tree and dirty-worktree preflight, path/metadata byte ceilings, immutable source materialization, Docker isolation, exact request/result binding, tests, public documentation, and standards claims. Do not interpret green coverage alone as security acceptance.

@coderabbitai

coderabbitai Bot commented Aug 5, 2026

Copy link
Copy Markdown
🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

repo="ContextualWisdomLab/noema"
pr="65"
requested="c604f929df89c387b1e28d6892247b2ce810e15d"

head_before="$(gh pr view "$pr" --repo "$repo" --json headRefOid --jq '.headRefOid')"
checked_out="$(git rev-parse HEAD)"
resolved_requested="$(git rev-parse "${requested}^{commit}")"

printf '%s\n' '== Exact revision state =='
printf 'checked_out_head=%s\nrequested_head=%s\npr_head_before=%s\n' \
  "$checked_out" "$resolved_requested" "$head_before"
git status --short

printf '%s\n' '== Changes in the PR =='
git diff --stat "${resolved_requested}^" "$resolved_requested"
git diff-tree --no-commit-id --name-status -r "$resolved_requested"

printf '%s\n' '== Current checks =='
gh pr view "$pr" --repo "$repo" --json reviewDecision,statusCheckRollup \
  --jq '{reviewDecision,checks:[.statusCheckRollup[] | {name,status,conclusion,detailsUrl}]}'

printf '%s\n' '== Candidate files =='
fd -t f -a . reviewer docs .github 2>/dev/null | \
  rg '(patch_validation|quarantined-patch-validation|CHANGELOG|doctor|workflow)' | sort

printf '%s\n' '== Production structure =='
ast-grep outline reviewer/noema_reviewer/patch_validation.py --items all --view signatures

printf '%s\n' '== Relevant production controls =='
rg -n -C 5 \
  'def (validate|inspect_patch_bytes|_verify_source_head|_materialize_committed_source|_preflight|_read_result_payload|_read_bounded|_run_git|_create_isolated_git_control|_validated_repository_path)|MAX_(PATCH|RESULT|TREE|ARCHIVE)|ls-tree|git archive|status --porcelain|TRUSTED_GIT_EXECUTABLE|GIT_NO_(REPLACE_OBJECTS|LAZY_FETCH)|O_NOFOLLOW|fstat|lstat|/output/result\.json|--network|--read-only|--user=|cap-drop|no-new-privileges|seccomp|--pull=never|--ulimit|fsize|DEVNULL|PASSED|returncode|head_sha|base_sha|patch_sha' \
  reviewer/noema_reviewer/patch_validation.py

printf '%s\n' '== Test coverage and contract references =='
rg -n -C 3 \
  '(patch_validation|ls-tree|archive|worktree|result\.json|Docker|MAX_RESULT|no newline|hunk|100%|docstring)' \
  reviewer/tests pyproject.toml pytest.ini setup.cfg .coveragerc 2>/dev/null || true

printf '%s\n' '== Documentation and changelog claims =='
rg -n -C 3 \
  '(patch validation|Patch validation|exact head|exact-head|credential|Docker|result\.json|16 KiB|64 MiB|ls-tree|archive|coverage|APA|NIST|SLSA)' \
  docs CHANGELOG.md README.md 2>/dev/null || true

printf '%s\n' '== Workflow privilege scan =='
rg -n -i -C 2 \
  '(contents:\s*write|permissions:.*write|github-token|COPILOT_GITHUB_TOKEN|repair[-_ ]?pr|self[-_ ]?modif|git push|gh pr)' \
  .github 2>/dev/null || true

printf '%s\n' '== Unresolved review threads =='
gh api graphql -F owner='ContextualWisdomLab' -F name='noema' -F number="$pr" -f query='
query($owner: String!, $name: String!, $number: Int!) {
  repository(owner: $owner, name: $name) {
    pullRequest(number: $number) {
      reviewThreads(first: 100) {
        nodes {
          isResolved
          path
          line
          comments(first: 1) { nodes { author { login } body } }
        }
      }
    }
  }
}' --jq '.data.repository.pullRequest.reviewThreads.nodes[] | select(.isResolved == false)' || true

head_after="$(gh pr view "$pr" --repo "$repo" --json headRefOid --jq '.headRefOid')"
printf '%s\n' '== Exact-head recheck =='
printf 'pr_head_after=%s\n' "$head_after"
if [ "$checked_out" != "$resolved_requested" ] ||
   [ "$head_before" != "$resolved_requested" ] ||
   [ "$head_after" != "$resolved_requested" ]; then
  printf '%s\n' 'EXACT_HEAD_MISMATCH_OR_CHANGED' >&2
  exit 2
fi

Length of output: 197417


🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '%s\n' '== Exact head and checks, compact =='
git rev-parse HEAD
gh pr view 65 --repo ContextualWisdomLab/noema --json headRefOid,reviewDecision,statusCheckRollup \
  --jq '{headRefOid,reviewDecision,checks:[.statusCheckRollup[] | {name,conclusion,status}]}'

printf '%s\n' '== Production symbol map =='
ast-grep outline reviewer/noema_reviewer/patch_validation.py --items all --view expanded

printf '%s\n' '== Function locations =='
rg -n \
  '^(def (validate|_validated_repository_path|_read_|_verify_source_head|_create_isolated_git_control|_isolated_git_environment|_preflight|_materialize_committed_source|_validate_|_build)|class (DockerPatchValidationRunner|PatchValidation))' \
  reviewer/noema_reviewer/patch_validation.py

printf '%s\n' '== Constants and subprocess calls =='
rg -n -C 4 \
  'MAX_|subprocess\.(run|Popen)|communicate\(|capture_output|stdout=|stderr=|timeout=|ls-tree|archive|status|Docker|docker|--network|--user|--mount|result\.json|RLIMIT_FSIZE|O_NOFOLLOW|os\.open|fstat|lstat' \
  reviewer/noema_reviewer/patch_validation.py

printf '%s\n' '== Focused test definitions =='
rg -n \
  '^\s*def test_.*(tree|archive|worktree|result|docker|hunk|path|metadata|output|timeout|descriptor|credential|materializ)' \
  reviewer/tests

printf '%s\n' '== Patch-validation file list =='
fd -t f -a . reviewer/tests | rg 'patch_validation' | sort

Length of output: 48137


🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '%s\n' '== Exact-tree and archive implementation =='
sed -n '838,1088p' reviewer/noema_reviewer/patch_validation.py

printf '%s\n' '== Runner and result implementation =='
sed -n '1110,1315p' reviewer/noema_reviewer/patch_validation.py

printf '%s\n' '== Prearchive and archive tests =='
sed -n '1,180p' reviewer/tests/test_patch_validation_prearchive_and_result_channel.py
sed -n '1,340p' reviewer/tests/test_patch_validation_archive_boundaries.py
sed -n '1,260p' reviewer/tests/test_patch_validation_exact_tree_and_output.py

printf '%s\n' '== Final live-head check =='
printf 'checked_out=%s\n' "$(git rev-parse HEAD)"
gh pr view 65 --repo ContextualWisdomLab/noema --json headRefOid --jq '.headRefOid'

Length of output: 41341


@seonghobae I completed a substantive review of exact head c604f929df89c387b1e28d6892247b2ce810e15d.

I found one blocking security issue. I do not approve this head.

Blocking: the exact-tree preflight is neither output-bounded nor bound to the archive.

  • Line 898 runs git ls-tree with stdout=subprocess.PIPE. Git can emit an arbitrarily large NUL-delimited path inventory. Python collects all output before _validated_exact_tree_output() applies the 20,000-member and aggregate-byte limits. A commit with many long paths can therefore consume host memory before validation fails.
  • Lines 861-896 validate the preflight output but discard object IDs, paths, and sizes.
  • Lines 936-1084 validate the archive and extracted snapshot only against a separate archive-derived manifest. They do not compare the archive file set, file sizes, or file content with the authenticated ls-tree inventory.

As a result, the implementation does not establish the documented “exact-tree and archive/extracted-manifest identity binding.” An archive that is structurally safe but omits a preflight file, adds a different file, or replaces a blob with equal-sized content can pass the current archive and extraction checks.

Use a bounded incremental ls-tree reader. Parse records as they arrive. Enforce a separate byte ceiling before accumulating data. Return an immutable inventory that includes canonical path, mode, object ID, and size. Require the archive manifest to match that inventory exactly. Verify each extracted regular file against the corresponding Git blob bytes or digest before Docker starts. Add regressions for output-byte overflow, archive omission/addition, equal-size substitution, and object-identity mismatch.

The completed checks and coverage evidence are positive, but they do not close this trust-boundary gap.

⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

Please review exact current head 9e03b17d0bdda75809f394aaafb0b44cf1267a51 against base 9adb784902cae19a150146a8fadd6a0491f67814. The prior rate-limited attempt did not start and must not be treated as review evidence. Treat findings anchored only to superseded heads as stale, while preserving any still-applicable issue. Focus on the streamed and explicitly bounded Git status/tree evidence, pre-archive source ceilings, descriptor-safe reads, patch path/hunk canonicality, archive/extraction equality, credential-free Docker isolation, exact request/result binding, test-first regressions, public documentation, and standards claims. Do not treat commit-status success as an approval or completed review.

@coderabbitai

coderabbitai Bot commented Aug 5, 2026

Copy link
Copy Markdown

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

repo="ContextualWisdomLab/noema"
pr="65"
base="9adb784902cae19a150146a8fadd6a0491f67814"
head="9e03b17d0bdda75809f394aaafb0b44cf1267a51"

pr_head_before="$(gh pr view "$pr" --repo "$repo" --json headRefOid --jq '.headRefOid')"
pr_base_before="$(gh pr view "$pr" --repo "$repo" --json baseRefOid --jq '.baseRefOid')"
checked_out="$(git rev-parse HEAD)"
resolved_base="$(git rev-parse "${base}^{commit}")"
resolved_head="$(git rev-parse "${head}^{commit}")"

printf '%s\n' '== Exact revision state =='
printf 'checked_out=%s\nrequested_base=%s\nrequested_head=%s\npr_base_before=%s\npr_head_before=%s\n' \
  "$checked_out" "$resolved_base" "$resolved_head" "$pr_base_before" "$pr_head_before"
git status --short

printf '%s\n' '== PR checks and review state =='
gh pr view "$pr" --repo "$repo" --json reviewDecision,statusCheckRollup \
  --jq '{reviewDecision, checks: [.statusCheckRollup[] | {name, status, conclusion, detailsUrl}]}'

printf '%s\n' '== Changed files against requested base =='
git diff --name-status "$resolved_base" "$resolved_head"
git diff --stat "$resolved_base" "$resolved_head"

printf '%s\n' '== Candidate files =='
fd -t f -a . | rg '(^|/)(patch_validation\.py|test_.*patch_validation.*\.py|CHANGELOG\.md|.*patch.validation.*\.md|.*doctor.*\.md)$' | sort

printf '%s\n' '== Production symbol outline =='
ast-grep outline reviewer/noema_reviewer/patch_validation.py --items all --view signatures

printf '%s\n' '== Relevant production definitions and controls =='
rg -n -C 5 \
  '^(def (_|)?(validate|inspect_patch_bytes|_validated_repository_path|_verify_source_head|_create_isolated_git_control|_isolated_git_environment|_verify_clean|_preflight|_materialize|_read|_run|_validate)|class )|git (status|ls-tree|archive)|Popen|PIPE|communicate|O_NOFOLLOW|fstat|lstat|MAX_(RESULT|PATCH|TREE|ARCHIVE)|GIT_NO_(REPLACE_OBJECTS|LAZY_FETCH)|--network|--read-only|--user=|cap-drop|no-new-privileges|--pull=never|result\.json|PASSED|returncode|head_sha|base_sha|patch_sha' \
  reviewer/noema_reviewer/patch_validation.py

printf '%s\n' '== Focused implementation sections =='
for range in 1,200 350,780 780,1150 1150,1450; do
  sed -n "${range}p" reviewer/noema_reviewer/patch_validation.py
done

printf '%s\n' '== Focused test declarations =='
rg -n -C 2 \
  '^\s*def test_.*(tree|archive|status|worktree|result|descriptor|hunk|path|metadata|docker|materializ|bound|timeout)' \
  reviewer/tests

printf '%s\n' '== Public documentation and changelog claims =='
rg -n -C 4 \
  '(quarantined patch|patch validation|exact.head|exact tree|ls-tree|archive|descriptor|result\.json|credential|Docker|16 KiB|64 MiB|NIST|SLSA|APA)' \
  README.md CHANGELOG.md docs 2>/dev/null || true

printf '%s\n' '== Workflow privilege and repair-workflow scan =='
rg -n -i -C 2 \
  '(contents:\s*write|permissions:.*write|COPILOT_GITHUB_TOKEN|repair[-_ ]?pr|self[-_ ]?modif|git push|gh pr)' \
  .github 2>/dev/null || true

printf '%s\n' '== Unresolved review threads =='
gh api graphql -F owner='ContextualWisdomLab' -F name='noema' -F number="$pr" -f query='
query($owner: String!, $name: String!, $number: Int!) {
  repository(owner: $owner, name: $name) {
    pullRequest(number: $number) {
      reviewThreads(first: 100) {
        nodes {
          isResolved
          path
          line
          comments(first: 1) { nodes { author { login } body } }
        }
      }
    }
  }
}' --jq '.data.repository.pullRequest.reviewThreads.nodes[] | select(.isResolved == false)' || true

pr_head_after="$(gh pr view "$pr" --repo "$repo" --json headRefOid --jq '.headRefOid')"
pr_base_after="$(gh pr view "$pr" --repo "$repo" --json baseRefOid --jq '.baseRefOid')"
printf '%s\n' '== Exact revision recheck =='
printf 'pr_base_after=%s\npr_head_after=%s\n' "$pr_base_after" "$pr_head_after"

if [ "$checked_out" != "$resolved_head" ] ||
   [ "$pr_base_before" != "$resolved_base" ] ||
   [ "$pr_head_before" != "$resolved_head" ] ||
   [ "$pr_base_after" != "$resolved_base" ] ||
   [ "$pr_head_after" != "$resolved_head" ]; then
  printf '%s\n' 'EXACT_REVISION_MISMATCH_OR_CHANGED' >&2
  exit 2
fi

Length of output: 200693


🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '%s\n' '== Current exact head and checks =='
git rev-parse HEAD
gh pr view 65 --repo ContextualWisdomLab/noema --json headRefOid,baseRefOid,reviewDecision,statusCheckRollup \
  --jq '{baseRefOid,headRefOid,reviewDecision,checks:[.statusCheckRollup[] | {name,status,conclusion}]}'

printf '%s\n' '== Source function map =='
rg -n \
  '^(def (_read|_run|_stream|_validate|_preflight|_materialize|_verify|_create|_build|validate)|class DockerPatchValidationRunner|class PatchValidation)' \
  reviewer/noema_reviewer/patch_validation.py

printf '%s\n' '== Exact-tree, archive, and extraction implementation =='
sed -n '760,1120p' reviewer/noema_reviewer/patch_validation.py

printf '%s\n' '== Git status, Git control, descriptor reads, and path/hunk parser =='
sed -n '250,760p' reviewer/noema_reviewer/patch_validation.py

printf '%s\n' '== Docker command, result channel, and request/result checks =='
sed -n '1120,1480p' reviewer/noema_reviewer/patch_validation.py

printf '%s\n' '== Focused patch-validation test files =='
fd -t f -a . reviewer/tests | rg 'patch_validation' | sort

printf '%s\n' '== Exact-tree and archive regression tests =='
for file in \
  reviewer/tests/test_patch_validation_prearchive_and_result_channel.py \
  reviewer/tests/test_patch_validation_archive_boundaries.py \
  reviewer/tests/test_patch_validation_exact_tree_and_output.py
do
  if [ -f "$file" ]; then
    printf '\n== %s ==\n' "$file"
    cat -n "$file"
  fi
done

printf '%s\n' '== Streaming, worktree, and diff-parser regression declarations =='
rg -n -C 3 \
  'def test_.*(stream|tree|status|dirty|worktree|archive|extract|result|descriptor|hunk|metadata|path|Docker|credential|timeout|terminate)' \
  reviewer/tests

printf '%s\n' '== Exact revision recheck =='
printf 'checked_out=%s\n' "$(git rev-parse HEAD)"
gh pr view 65 --repo ContextualWisdomLab --json headRefOid,baseRefOid \
  --jq '{baseRefOid,headRefOid}'

Length of output: 155700


@seonghobae I completed a substantive review of exact head 9e03b17d0bdda75809f394aaafb0b44cf1267a51 against base 9adb784902cae19a150146a8fadd6a0491f67814.

I found one blocking issue. I do not approve this head.

Blocking: the exact-tree preflight is not bound to the archive or extracted source bytes.

  • Line 940 validates each git ls-tree object ID, path, mode, and size. It retains only aggregate counters.
  • Line 1050 runs the bounded preflight before git archive. This correctly prevents an oversized tree from reaching archive allocation.
  • Line 1166 then creates the archive without an immutable tree inventory.
  • Line 1084 creates expected_entries from the archive itself. The entries contain only path, type, and size.
  • Line 1144 compares the extracted snapshot only to that archive-derived path/type/size map.

The code does not compare archive members to the authenticated ls-tree paths, modes, object IDs, or blob bytes. It therefore does not establish the documented exact-tree/archive/extraction equality.

A structurally valid archive can omit a preflight path, add a path, or replace a blob with equal-size content. The current archive and extraction checks can accept it because both checks use the archive-derived manifest.

Retain an immutable exact-tree inventory from the streamed preflight. The inventory should contain each canonical path, mode, object ID, and size. Require the archive file set and sizes to match that inventory exactly. Before Docker starts, verify each extracted regular file against its authenticated Git blob bytes or a trusted digest derived from that blob.

Add regressions for:

  1. An archive member omitted from the exact-tree inventory.
  2. An archive member added after the preflight.
  3. Equal-size replacement content.
  4. Object-ID mismatch.
  5. Extraction bytes that differ from the authenticated blob.

The streamed status and tree readers now use bounded incremental reads. The descriptor-safe result-file boundary and Docker controls also appear materially improved. Those controls do not close the missing source-byte binding.

You are interacting with an AI system.

Copy link
Copy Markdown
Contributor Author

@opencode-agent @cwl-noema-review

Independent review is requested for exact current head 9e03b17d0bdda75809f394aaafb0b44cf1267a51 against base 9adb784902cae19a150146a8fadd6a0491f67814. Submit a formal GitHub APPROVE only after verifying the exact head, successful required workflows, streamed bounded Git evidence, patch canonicality, archive/extraction equality, credential-free Docker isolation, structured evidence binding, tests, documentation, and repository policy. A mention, comment, commit status, self-review, predecessor-head decision, or rate-limited automated response does not satisfy the independent-approval gate.

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

Exact-head request: base 9adb784902cae19a150146a8fadd6a0491f67814, head 9e03b17d0bdda75809f394aaafb0b44cf1267a51. Please review only this exact head. A queued, pending, rate-limited, or commit-status-only result is not review completion.

@coderabbitai

coderabbitai Bot commented Aug 5, 2026

Copy link
Copy Markdown

@seonghobae I will review PR #65 only if the live base and head match the requested revisions. I will not treat a queued, pending, rate-limited, or commit-status-only result as review completion.

⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

Please perform the substantive review of exact current head 9e03b17d0bdda75809f394aaafb0b44cf1267a51 now that the reported cooldown has elapsed. Bind findings only to base 9adb784902cae19a150146a8fadd6a0491f67814 and this head; do not treat the existing CodeRabbit commit-status success or prior rate-limit notice as review evidence. Focus on streamed Git status/tree bounds, exact-commit archive and extraction equality, strict patch metadata/path identity, descriptor-safe bounded reads, credential-free Docker isolation, structured request/result rebinding, tests, documentation, and scope claims.

@coderabbitai

coderabbitai Bot commented Aug 5, 2026

Copy link
Copy Markdown

Your plan includes PR reviews subject to rate limits. More reviews will be available in 20 seconds.

Copy link
Copy Markdown
Contributor Author

@opencode-agent

Implement the valid blocking CodeRabbit finding on exact current head 9e03b17d0bdda75809f394aaafb0b44cf1267a51; re-read the live PR head immediately before every write and refuse or re-plan if it moved.

Add RED regressions first for mutation or substitution between the streamed ls-tree preflight and git archive: the preflight inventory must be cryptographically and structurally bound to the archive and extracted snapshot, not merely used for aggregate bounds. Preserve a bounded immutable inventory containing each canonical path, regular mode, exact blob size, and object ID. Require the archive to have the same file set, types, modes, sizes, and Git blob identities; recompute the repository-format object digest from extracted regular-file bytes (or use an equivalently bounded, configuration-isolated verification) and reject any mismatch before Docker receives the source. Keep the 20,000-member, 4 KiB path, 16 MiB metadata, 64 MiB per-file, 512 MiB aggregate, timeout, early-termination, credential isolation, and exact-head refusal limits. Ensure directory synthesis cannot conceal file-set drift.

Cover at least: same-size blob substitution, different-size substitution, added/removed/renamed archive paths, mode drift, object-ID mismatch, tree/archive race, malformed or duplicate inventory entries, and all new failure/cleanup branches. Maintain 100% production statement and branch coverage plus 100% public docstring coverage. Update authoritative documentation and CHANGELOG.md so claims describe tree-to-archive-to-extracted-byte identity rather than archive-self-consistency. Do not add, restore, or use any write-capable repair workflow, self-modifying Action, .github/workflows/repair-* file, or change to the existing review-agent key contract. After the fix, rerun exact-head ci, reviewer-ci, Security Scan, and request a fresh CodeRabbit review and eligible independent approval for the new head.

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

Exact-head review request: base 9adb784902cae19a150146a8fadd6a0491f67814, head 9e03b17d0bdda75809f394aaafb0b44cf1267a51. Please perform a substantive review of this exact head. A commit-status success without completed review evidence is not sufficient; queued, pending, skipped, or rate-limited review is not success.

@coderabbitai

coderabbitai Bot commented Aug 5, 2026

Copy link
Copy Markdown

@seonghobae I will review PR #65 only if the live base and head match the requested revisions.

I will treat a queued, pending, skipped, rate-limited, or commit-status-only response as incomplete review evidence.

⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copy link
Copy Markdown
Contributor Author

@opencode-agent

Implement the valid exact-head security finding on current head 89b4e5c40e6bdd7eab4f364ccd9161870cf68390; refuse every write if the PR head moves.

The RED regression test_trailing_mode_tokens_cannot_hide_a_symlink_mode is now committed and reviewer-ci fails as intended. inspect_patch_bytes currently accepts new file mode 120000 100644 because the special-mode regex requires end-of-line while the later parser only checks whether the line ends in 100644 or 100755. Git accepts this header and materializes a symlink, so this is a fail-open quarantine bypass.

Fix test-first by parsing every old mode, new mode, new file mode, and deleted file mode line with one exact full-match grammar containing exactly one six-digit mode token. Reject malformed/trailing-token metadata distinctly; reject 120000 and 160000; allow only 100644 and 100755. Do not add a shadow module, import-time monkeypatch, repair workflow, self-modifying workflow, or any contents-write workflow workaround.

Update docs/quarantined-patch-validation.md, docs/doctoring/quarantined-patch-validation.md, and CHANGELOG.md to state that mode metadata is exact-token parsed and ambiguity/trailing tokens fail closed. Preserve beginner-readable public docstrings, 100% production statement/branch coverage, and 100% public-docstring coverage. Rerun all exact-head checks, then request current-head CodeRabbit and eligible independent approval. Do not merge, bump, or release.

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

Review exact current head 89b4e5c40e6bdd7eab4f364ccd9161870cf68390 against base 9adb784902cae19a150146a8fadd6a0491f67814. Treat the committed RED regression and failing reviewer-ci as an unresolved blocking security finding, not success. Focus on exact-token parsing for every Git mode metadata line, symlink/gitlink refusal, trailing-token ambiguity, patch path/hunk integrity, bounded evidence channels, stale-head refusal, least privilege, and the absence of repair/self-modifying/write-capable workflows. Do not approve or report completion unless reviewing this exact head.

@coderabbitai

coderabbitai Bot commented Aug 5, 2026

Copy link
Copy Markdown

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

repo="ContextualWisdomLab/noema"
pr="65"
base="9adb784902cae19a150146a8fadd6a0491f67814"
head="89b4e5c40e6bdd7eab4f364ccd9161870cf68390"

pr_base_before="$(gh pr view "$pr" --repo "$repo" --json baseRefOid --jq '.baseRefOid')"
pr_head_before="$(gh pr view "$pr" --repo "$repo" --json headRefOid --jq '.headRefOid')"
checked_out="$(git rev-parse HEAD)"
resolved_base="$(git rev-parse "${base}^{commit}")"
resolved_head="$(git rev-parse "${head}^{commit}")"

printf '%s\n' '== Exact revision state =='
printf 'checked_out=%s\nrequested_base=%s\nrequested_head=%s\npr_base_before=%s\npr_head_before=%s\n' \
  "$checked_out" "$resolved_base" "$resolved_head" "$pr_base_before" "$pr_head_before"
git status --short

printf '%s\n' '== PR changes and checks =='
git diff --name-status "$resolved_base" "$resolved_head"
gh pr view "$pr" --repo "$repo" --json reviewDecision,statusCheckRollup \
  --jq '{reviewDecision,checks:[.statusCheckRollup[] | {name,status,conclusion,detailsUrl}]}'

printf '%s\n' '== Production structure =='
ast-grep outline reviewer/noema_reviewer/patch_validation.py --items all --view signatures

printf '%s\n' '== Mode metadata parser and patch integrity implementation =='
rg -n -C 10 \
  '(old mode|new mode|new file mode|deleted file mode|inspect_patch_bytes|_validated_repository_path|index |diff --git|@@ |no newline|120000|160000|100644|100755|fullmatch|match\()' \
  reviewer/noema_reviewer/patch_validation.py

printf '%s\n' '== Bounded evidence and Docker isolation implementation =='
rg -n -C 6 \
  '(git status|ls-tree|Popen|PIPE|O_NOFOLLOW|fstat|MAX_|--network|--read-only|--user=|cap-drop|no-new-privileges|--pull=never|result\.json|DEVNULL|GIT_NO_)' \
  reviewer/noema_reviewer/patch_validation.py

printf '%s\n' '== Focused source sections =='
rg -n '^(def (inspect_patch_bytes|_validated_repository_path|_verify_source_head|_preflight|_materialize|_read|_run)|class )' \
  reviewer/noema_reviewer/patch_validation.py
sed -n '250,780p' reviewer/noema_reviewer/patch_validation.py
sed -n '780,1480p' reviewer/noema_reviewer/patch_validation.py

printf '%s\n' '== Focused regression tests =='
rg -n -C 5 \
  '(trailing_mode_tokens|mode.*symlink|symlink|gitlink|old mode|new mode|new file mode|deleted file mode|hunk|descriptor|result|Docker|worktree|ls-tree)' \
  reviewer/tests

printf '%s\n' '== Public contract and workflow scan =='
rg -n -C 4 \
  '(mode metadata|trailing|symlink|gitlink|patch validation|exact-token|credential|Docker|result\.json)' \
  docs CHANGELOG.md README.md 2>/dev/null || true
rg -n -i -C 3 \
  '(contents:\s*write|permissions:.*write|repair[-_ ]?pr|self[-_ ]?modif|git push|gh pr|COPILOT_GITHUB_TOKEN)' \
  .github 2>/dev/null || true

printf '%s\n' '== Unresolved review threads =='
gh api graphql -F owner='ContextualWisdomLab' -F name='noema' -F number="$pr" -f query='
query($owner: String!, $name: String!, $number: Int!) {
  repository(owner: $owner, name: $name) {
    pullRequest(number: $number) {
      reviewThreads(first: 100) {
        nodes {
          isResolved
          path
          line
          comments(first: 1) { nodes { author { login } body } }
        }
      }
    }
  }
}' --jq '.data.repository.pullRequest.reviewThreads.nodes[] | select(.isResolved == false)' || true

pr_base_after="$(gh pr view "$pr" --repo "$repo" --json baseRefOid --jq '.baseRefOid')"
pr_head_after="$(gh pr view "$pr" --repo "$repo" --json headRefOid --jq '.headRefOid')"
printf '%s\n' '== Exact revision recheck =='
printf 'pr_base_after=%s\npr_head_after=%s\n' "$pr_base_after" "$pr_head_after"

if [ "$checked_out" != "$resolved_head" ] ||
   [ "$pr_base_before" != "$resolved_base" ] ||
   [ "$pr_head_before" != "$resolved_head" ] ||
   [ "$pr_base_after" != "$resolved_base" ] ||
   [ "$pr_head_after" != "$resolved_head" ]; then
  echo 'EXACT_REVISION_MISMATCH_OR_CHANGED' >&2
  exit 2
fi

Length of output: 197658


🏁 Script executed:

#!/bin/bash
set -euo pipefail

repo="ContextualWisdomLab/noema"
pr="65"
base="9adb784902cae19a150146a8fadd6a0491f67814"
head="89b4e5c40e6bdd7eab4f364ccd9161870cf68390"

printf '%s\n' '== Exact revision precheck =='
git rev-parse HEAD
gh pr view "$pr" --repo "$repo" --json baseRefOid,headRefOid \
  --jq '{baseRefOid,headRefOid}'

printf '%s\n' '== Mode parser definitions and nearby source =='
rg -n \
  '(MODE|mode|inspect_patch_bytes|_parse|_validate.*metadata|new file mode|deleted file mode|old mode|new mode)' \
  reviewer/noema_reviewer/patch_validation.py

for line in $(rg -n \
  '(new file mode|deleted file mode|old mode|new mode|def inspect_patch_bytes)' \
  reviewer/noema_reviewer/patch_validation.py | cut -d: -f1 | sort -nu); do
  start=$((line - 18))
  [ "$start" -lt 1 ] && start=1
  end=$((line + 30))
  printf '\n== reviewer/noema_reviewer/patch_validation.py:%s-%s ==\n' "$start" "$end"
  sed -n "${start},${end}p" reviewer/noema_reviewer/patch_validation.py
done

printf '%s\n' '== Exact committed RED regression and related mode tests =='
rg -n -C 12 \
  '(trailing_mode_tokens_cannot_hide_a_symlink_mode|new file mode|deleted file mode|old mode|new mode|120000|160000|trailing)' \
  reviewer/tests

printf '%s\n' '== Files changed in this PR under workflows =='
git diff --name-status "$base" "$head" -- .github/workflows
git diff -- "$base" "$head" -- .github/workflows | \
  rg -n -i -C 3 '(repair|self.modif|workflow_dispatch|contents:\s*write|git push|gh pr|curl .*api\.github)' || true

printf '%s\n' '== Repair workflow and transient mutator inventory =='
fd -t f -a .github/workflows reviewer -E .git | \
  rg -i '(repair|patch.validation.*repair|mutat)' || true
rg -n -i -C 2 \
  '(repair[-_ ]?pr|self[-_ ]?modif|self[-_ ]?delet|git\s+commit|git\s+push|gh\s+pr)' \
  .github/workflows 2>/dev/null || true

printf '%s\n' '== Current check failures only =='
gh pr view "$pr" --repo "$repo" --json statusCheckRollup \
  --jq '.statusCheckRollup[] | select(.conclusion != "SUCCESS") | {name,status,conclusion,detailsUrl}'

printf '%s\n' '== Exact revision postcheck =='
printf 'checked_out=%s\n' "$(git rev-parse HEAD)"
gh pr view "$pr" --repo "$repo" --json baseRefOid,headRefOid \
  --jq '{baseRefOid,headRefOid}'

Length of output: 32417


@seonghobae I completed the review of base 9adb784902cae19a150146a8fadd6a0491f67814 and head 89b4e5c40e6bdd7eab4f364ccd9161870cf68390.

I found one blocking issue. I do not approve this head.

Blocking: mode metadata accepts a trailing-token symlink bypass.

reviewer/noema_reviewer/patch_validation.py defines PATCH_MODE_PATTERN for only an exact special-mode line. The parser later handles old mode, new mode, new file mode, and deleted file mode metadata with this suffix test:

if not line.endswith((" 100644", " 100755")):
    raise ValueError("patch contains an unsupported file mode")

This accepts:

new file mode 120000 100644

The special-mode expression does not match because the line has an extra token. The suffix test then accepts the trailing 100644. Git can interpret the first mode token as 120000 and materialize a symlink.

The committed regression in reviewer/tests/test_patch_validation_mode_boundaries.py::test_trailing_mode_tokens_cannot_hide_a_symlink_mode proves this condition. It expects malformed mode metadata. The current reviewer check fails.

Parse each mode metadata line with one fullmatch grammar. The grammar must allow exactly one six-digit mode token. Reject malformed or trailing-token metadata. Reject 120000 and 160000. Permit only 100644 and 100755.

The live base and head remained unchanged during inspection. The failing reviewer check confirms that this RED regression remains unresolved.

You are interacting with an AI system.

Copy link
Copy Markdown
Contributor Author

@opencode-agent

Apply the already-validated minimal production fix only if PR #65 still has exact head 89b4e5c40e6bdd7eab4f364ccd9161870cf68390; abort without writing if it moved.

  1. Replace the multiline special-mode pre-scan with one full-line grammar:
MODE_METADATA_PATTERN = re.compile(
    r"^(?:old mode|new mode|new file mode|deleted file mode) (?P<mode>[0-9]{6})$"
)
  1. Remove PATCH_MODE_PATTERN.search(text). In the existing per-line mode-metadata branch, after the placement check, use MODE_METADATA_PATTERN.fullmatch(line). A non-match must raise ValueError("patch contains malformed mode metadata"); 120000/160000 must raise the existing symlink-or-gitlink error; values other than 100644/100755 must raise the existing unsupported-file-mode error.

This preserves distinct malformed, special-object, and unsupported-regular-mode semantics and closes the Git-accepted new file mode 120000 100644 trailing-token bypass. The committed RED test already demonstrates the issue; retain exact special-mode, unsupported-mode, regular-mode, and misplaced-mode regressions and restore 100% production statement/branch and public-docstring coverage.

Update docs/quarantined-patch-validation.md, docs/doctoring/quarantined-patch-validation.md, and CHANGELOG.md to state that every old mode/new mode/new file mode/deleted file mode line is full-match parsed as exactly one six-digit token and ambiguous trailing tokens fail closed. Do not create or use repair workflows, self-modifying Actions, branch-patching workflows, or any write-capable .github/workflows/repair-* mechanism. Rerun the exact-head reviewer tests and checks after committing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants