fix(coverage): scope changed JavaScript evidence to runtime code - #765
fix(coverage): scope changed JavaScript evidence to runtime code#765seonghobae wants to merge 7 commits into
Conversation
|
Warning Review limit reached
Next review available in: 4 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (6)
Comment |
|
@opencode-agent @cwl-noema-review Please independently review exact current head |
![opencode-agent[bot]](https://avatars.githubusercontent.com/in/1549082?s=60&v=4){kind=small}
There was a problem hiding this comment.
Pull request overview
OpenCode cannot approve yet because required coverage evidence did not pass.
Review outcome
1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
-
Problem: The required coverage-evidence job result was
failure, so OpenCode cannot establish approval sufficiency for this head. -
Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.
-
Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports
successwith required evidence or explicit no-source not-applicable evidence. -
Regression test: Keep the approval branch checking
needs.coverage-evidence.result == successbefore posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present. -
Result: REQUEST_CHANGES
-
Reason: coverage-evidence result was
failure, so required test/docstring evidence was not proven for current headc216f0cb4d680c381c4389d9b04195d42d1be57d. -
Head SHA:
c216f0cb4d680c381c4389d9b04195d42d1be57d -
Workflow run: 30967466564
-
Workflow attempt: 1
Coverage evidence
Coverage Decision
- Result: FAIL
- Test evidence: not proven passing
- Docstring evidence: not proven passing when configured
- Failure count: 1
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Docs: javascript-runtime-coverage-scope.md"]
S1 --> I1["operator or user guidance"]
I1 --> R1["Review risk: Docs: javascript-runtime-coverage-scope.md"]
R1 --> V1["docs review"]
Evidence --> S2["Changed file (2 files)"]
S2 --> I2["repository behavior"]
I2 --> R2["Review risk: Changed file (2 files)"]
R2 --> V2["required checks"]
Evidence --> S3["CI script: javascript_coverage_gate.py"]
S3 --> I3["review and security gate shell path"]
I3 --> R3["Review risk: CI script: javascript_coverage_gate.py"]
R3 --> V3["bash -n plus Strix self-test"]
Evidence --> S4["Test: test_javascript_coverage_scope.py"]
S4 --> I4["regression suite"]
I4 --> R4["Review risk: Test: test_javascript_coverage_scope.py"]
R4 --> V4["targeted test run"]
OpenCode Review Overview
Pull request overviewOpenCode cannot approve yet because required coverage evidence did not pass. Review outcome1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
Coverage evidenceCoverage Decision
Changed-File Evidence Mapflowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Docs: javascript-runtime-coverage-scope.md"]
S1 --> I1["operator or user guidance"]
I1 --> R1["Review risk: Docs: javascript-runtime-coverage-scope.md"]
R1 --> V1["docs review"]
Evidence --> S2["Changed file (2 files)"]
S2 --> I2["repository behavior"]
I2 --> R2["Review risk: Changed file (2 files)"]
R2 --> V2["required checks"]
Evidence --> S3["CI script: javascript_coverage_gate.py"]
S3 --> I3["review and security gate shell path"]
I3 --> R3["Review risk: CI script: javascript_coverage_gate.py"]
R3 --> V3["bash -n plus Strix self-test"]
Evidence --> S4["Test: test_javascript_coverage_scope.py"]
S4 --> I4["regression suite"]
I4 --> R4["Review risk: Test: test_javascript_coverage_scope.py"]
R4 --> V4["targeted test run"]
|
|
Root-cause repair at exact head The failed OpenCode coverage evidence was reproduced from its exact merge-tree artifact. The focused JavaScript runtime/tooling classifier tests were not the failing surface; the suite stopped at No JavaScript coverage rule, product-code fail-closed behavior, security threshold, or reviewer credential was weakened. Fresh exact-head coverage, security, independent review, and branch protection remain required. |
|
@opencode-agent @cwl-noema-review Please independently review exact current head |
|
@coderabbitai review Please review exact current head @opencode-agent Please rerun the complete exact-head coverage, test, and docstring evidence on @cwl-noema-review Please independently review exact head |
|
|
|
@coderabbitai review Please review exact current head |
|
I will assess only current actionable findings. I will not use evidence from the superseded head.
|
|
@opencode-agent @cwl-noema-review Please independently review exact current head |
|
Review-only exact-head dispatch for @cwl-noema-review Please independently review the same exact head after verifying all current-head gates. |
|
@opencode-agent @cwl-noema-review Please independently review exact current head |
|
@coderabbitai full review |
|
|
@opencode-agent @cwl-noema-review @coderabbitai review Independently review exact current head |
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
head='33d6168de6aef22f8a76dd19fcc88d99f3613d9e'
base='3f65dbee6672b78802e7d71d49c390f3817bb03b'
printf '%s\n' '== repository head =='
git rev-parse HEAD
git cat-file -e "${head}^{commit}"
git cat-file -e "${base}^{commit}"
printf '%s\n' '== changed files =='
git diff --name-status "$base" "$head"
printf '%s\n' '== relevant file candidates =='
fd -t f . | rg '(^|/)(javascript_coverage_gate\.py|test_.*coverage.*\.py|test_.*codeql.*\.py|.*doctoring.*\.md|.*(?:requirements|lock).*|.*codeql.*\.(?:yml|yaml)|.*scheduled-security-scan.*\.(?:yml|yaml))$' || true
printf '%s\n' '== source and test outlines =='
for f in $(fd -t f 'javascript_coverage_gate\.py|test_.*coverage.*\.py|test_.*codeql.*\.py' .); do
echo "--- $f"
ast-grep outline "$f" --items all --view compact || true
done
printf '%s\n' '== classifier, CodeQL revision, lock floor, and APA reference evidence =='
rg -n -i -C 3 \
'config|check-|verify-|coverage-final|statement|branch|function|line|aiohttp==|cryptography==|d1ba80a13dd99fba24a470575428917156a28b43|CodeQL|APA|Vitest|NIST|SSDF' \
scripts tests .github docs requirements* 2>/dev/null || true
printf '%s\n' '== full relevant diff =='
git diff --find-renames --find-copies --unified=80 "$base" "$head" -- \
'scripts/ci/javascript_coverage_gate.py' \
'tests/**' \
'.github/workflows/**' \
'docs/doctoring/**' \
'*requirements*' \
'*lock*' || trueLength of output: 161474 🏁 Script executed: #!/bin/bash
set -euo pipefail
head='33d6168de6aef22f8a76dd19fcc88d99f3613d9e'
base='3f65dbee6672b78802e7d71d49c390f3817bb03b'
printf '%s\n' '== exact commits and changed paths =='
git rev-parse HEAD
git diff --name-status "$base" "$head"
printf '%s\n' '== coverage gate: complete current implementation =='
cat -n scripts/ci/javascript_coverage_gate.py
printf '%s\n' '== coverage-scope tests: test names and complete file =='
rg -n '^\s*def test_|^\s*`@pytest`\.mark' tests/test_javascript_coverage_scope.py
cat -n tests/test_javascript_coverage_scope.py
printf '%s\n' '== tests that exercise the coverage gate =='
rg -n -C 2 \
'javascript_coverage_gate|is_runtime_source|changed_runtime_lines|coverage-final\.json|is_repository_verification_script|TOOL_CONFIG_NAME_RE' \
tests -g '*.py'
printf '%s\n' '== source lock and generated-lock integrity controls =='
rg -n -C 3 \
'requirements-strix-ci|generate-hashes|uv pip compile|lock integrity|aiohttp==|cryptography==' \
.github scripts tests README.md requirements-strix-ci.txt requirements-strix-ci-hashes.txt \
-g '!requirements-strix-ci-hashes.txt'
printf '%s\n' '== changed direct requirements and lock package entries =='
cat -n requirements-strix-ci.txt
rg -n -A 4 -B 1 '^(aiohttp|cryptography|pyopenssl)==' requirements-strix-ci-hashes.txt
printf '%s\n' '== CodeQL action use in changed workflow and contract tests =='
rg -n -C 3 \
'github/codeql-action/(init|analyze|upload-sarif)@|test_codeql_action_steps_use_one_version_per_workflow' \
.github/workflows/scheduled-security-scan.yml tests -g '*.py'
printf '%s\n' '== doctoring references =='
sed -n '1,220p' docs/doctoring/javascript-runtime-coverage-scope.md
printf '%s\n' '== exact-head PR state and checks =='
gh pr view 765 --repo ContextualWisdomLab/.github \
--json headRefOid,baseRefOid,reviewDecision,mergeStateStatus,statusCheckRollup \
--jq '{headRefOid,baseRefOid,reviewDecision,mergeStateStatus,checks:[.statusCheckRollup[]|{name,conclusion,status,headSha}]}'Length of output: 80385
I verified these items on head
I cannot approve yet. The exact-head You are interacting with an AI system. |
|
@cwl-noema-review @opencode-agent Please perform fresh independent review of exact current head |
|
@coderabbitai review @opencode-agent @cwl-noema-review Please independently review exact current head |
🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
head='33d6168de6aef22f8a76dd19fcc88d99f3613d9e'
base='3f65dbee6672b78802e7d71d49c390f3817bb03b'
printf '%s\n' '== exact revision =='
git rev-parse HEAD
git cat-file -e "${head}^{commit}"
git cat-file -e "${base}^{commit}"
printf '%s\n' '== changed paths =='
git diff --name-status "$base" "$head"
printf '%s\n' '== focused file discovery =='
fd -t f . | rg '(^|/)(javascript_coverage_gate\.py|test_javascript_coverage_scope\.py|test_codeql_pr_workflow_contract\.py|scheduled-security-scan\.ya?ml|javascript-runtime-coverage-scope\.md|requirements-strix-ci(?:-hashes)?\.txt)$' || true
printf '%s\n' '== coverage gate structure =='
ast-grep outline scripts/ci/javascript_coverage_gate.py --items all --view expanded || true
printf '%s\n' '== coverage classifier and evidence enforcement =='
rg -n -C 5 \
'TOOL_CONFIG|is_runtime_source|is_repository_verification_script|coverage-final|statement|branch|function|line|scripts|src' \
scripts/ci/javascript_coverage_gate.py tests/test_javascript_coverage_scope.py
printf '%s\n' '== scope test names =='
rg -n '^\s*def test_' tests/test_javascript_coverage_scope.py
printf '%s\n' '== CodeQL immutable revisions =='
rg -n -C 2 \
'github/codeql-action/(init|analyze|upload-sarif)@|d1ba80a13dd99fba24a470575428917156a28b43' \
.github/workflows/scheduled-security-scan.yml tests/test_codeql_pr_workflow_contract.py
printf '%s\n' '== Strix direct requirements and lock entries =='
rg -n -C 2 \
'^(aiohttp|cryptography)==|generate|hash|integrity' \
requirements-strix-ci.txt requirements-strix-ci-hashes.txt tests .github scripts 2>/dev/null || true
printf '%s\n' '== doctoring citations and stated boundaries =='
rg -n -i -C 2 \
'APA|Vitest|NIST|SSDF|config|check-|verify-|src|scripts|fail-closed|Inkspan|100%' \
docs/doctoring/javascript-runtime-coverage-scope.md
printf '%s\n' '== PR head and checks =='
gh pr view 765 --repo ContextualWisdomLab/.github \
--json headRefOid,baseRefOid,reviewDecision,mergeStateStatus,statusCheckRollup \
--jq '{headRefOid,baseRefOid,reviewDecision,mergeStateStatus,checks:[.statusCheckRollup[]|{name,conclusion,status,headSha}]}'Length of output: 202670 🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '%s\n' '== exact classifier implementation =='
nl -ba scripts/ci/javascript_coverage_gate.py | sed -n '1,280p'
printf '%s\n' '== regression test source, first section =='
nl -ba tests/test_javascript_coverage_scope.py | sed -n '1,420p'
printf '%s\n' '== regression test source, remaining section =='
nl -ba tests/test_javascript_coverage_scope.py | sed -n '421,840p'
printf '%s\n' '== CodeQL scheduled workflow action steps =='
rg -n -C 5 'github/codeql-action/(init|analyze|upload-sarif)@' \
.github/workflows/scheduled-security-scan.yml
printf '%s\n' '== CodeQL revision contract =='
rg -n -C 5 'test_codeql_action_steps_use_one_version_per_workflow|codeql-action' \
tests/test_codeql_pr_workflow_contract.py
printf '%s\n' '== direct dependency and generated lock entries =='
rg -n -A 8 -B 2 '^(aiohttp|cryptography)==' \
requirements-strix-ci.txt requirements-strix-ci-hashes.txt
printf '%s\n' '== changed documentation complete =='
nl -ba docs/doctoring/javascript-runtime-coverage-scope.md | sed -n '1,190p'
printf '%s\n' '== diff limited to reviewed artifacts =='
git diff --unified=20 3f65dbee6672b78802e7d71d49c390f3817bb03b \
33d6168de6aef22f8a76dd19fcc88d99f3613d9e -- \
scripts/ci/javascript_coverage_gate.py \
tests/test_javascript_coverage_scope.py \
tests/test_codeql_pr_workflow_contract.py \
.github/workflows/scheduled-security-scan.yml \
requirements-strix-ci.txt \
requirements-strix-ci-hashes.txt \
docs/doctoring/javascript-runtime-coverage-scope.mdLength of output: 1921 🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '%s\n' '== classifier definitions and gate enforcement =='
rg -n -C 8 \
'TOOL_CONFIG|KNOWN_TOOL|is_runtime_source|is_repository_verification_script|def main|coverage-final|missing.*coverage|statements|branches|functions|lines' \
scripts/ci/javascript_coverage_gate.py
printf '%s\n' '== exact classifier file =='
sed -n '1,330p' scripts/ci/javascript_coverage_gate.py
printf '%s\n' '== regression case names and relevant assertions =='
rg -n -C 6 \
'Inkspan|autosave|vite\.autosave|verify-framework|verify-package|feature\.config|serve-package|verify-session|nested|missing.*coverage|statements|branches|functions|lines|docstring' \
tests/test_javascript_coverage_scope.py
printf '%s\n' '== test function inventory =='
rg -n '^[[:space:]]*def test_' tests/test_javascript_coverage_scope.py
printf '%s\n' '== scheduled CodeQL action revisions =='
rg -n -C 3 'github/codeql-action/(init|analyze|upload-sarif)@' \
.github/workflows/scheduled-security-scan.yml
printf '%s\n' '== CodeQL revision contract test =='
rg -n -C 8 'test_codeql_action_steps_use_one_version_per_workflow|ACTION|revision|codeql-action' \
tests/test_codeql_pr_workflow_contract.py
printf '%s\n' '== Strix direct and lock package records =='
rg -n -A 6 -B 2 '^(aiohttp|cryptography)==' \
requirements-strix-ci.txt requirements-strix-ci-hashes.txt
printf '%s\n' '== doctoring references =='
sed -n '1,170p' docs/doctoring/javascript-runtime-coverage-scope.mdLength of output: 47383
I verified the following:
GitHub still reports
|
|
Closing as superseded by consolidated PR #757. The production classifier and regression suite are already preserved there byte-for-byte: |
Pull request was closed
|
@coderabbitai review @opencode-agent @cwl-noema-review Re-evaluate exact head |
🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
head='33d6168de6aef22f8a76dd19fcc88d99f3613d9e'
base='3f65dbee6672b78802e7d71d49c390f3817bb03b'
printf '%s\n' '== exact revisions =='
git rev-parse HEAD
git cat-file -e "${head}^{commit}"
git cat-file -e "${base}^{commit}"
printf '%s\n' '== changed paths =='
git diff --name-status "$base" "$head"
printf '%s\n' '== coverage gate structure =='
ast-grep outline scripts/ci/javascript_coverage_gate.py --items all --view expanded
printf '%s\n' '== coverage gate implementation =='
sed -n '1,360p' scripts/ci/javascript_coverage_gate.py
printf '%s\n' '== coverage-scope test inventory =='
rg -n '^[[:space:]]*def test_' tests/test_javascript_coverage_scope.py
printf '%s\n' '== bounded classifier and evidence assertions =='
rg -n -C 5 \
'Inkspan|autosave|vite\.autosave|verify-framework|verify-package|feature\.config|serve-package|verify-session|nested|missing.*coverage|statements|branches|functions|lines|docstring|temporary' \
tests/test_javascript_coverage_scope.py
printf '%s\n' '== CodeQL action revisions and contract =='
rg -n -C 4 \
'github/codeql-action/(init|analyze|upload-sarif)@|test_codeql_action_steps_use_one_version_per_workflow' \
.github/workflows/scheduled-security-scan.yml tests/test_codeql_pr_workflow_contract.py
printf '%s\n' '== Strix dependency pins =='
rg -n -A 7 -B 2 '^(aiohttp|cryptography)==' \
requirements-strix-ci.txt requirements-strix-ci-hashes.txt
printf '%s\n' '== doctoring record =='
sed -n '1,220p' docs/doctoring/javascript-runtime-coverage-scope.md
printf '%s\n' '== reviewed diff =='
git diff --unified=30 "$base" "$head" -- \
scripts/ci/javascript_coverage_gate.py \
tests/test_javascript_coverage_scope.py \
tests/test_codeql_pr_workflow_contract.py \
.github/workflows/scheduled-security-scan.yml \
requirements-strix-ci.txt \
requirements-strix-ci-hashes.txt \
docs/doctoring/javascript-runtime-coverage-scope.md
printf '%s\n' '== current PR checks and decision =='
gh pr view 765 --repo ContextualWisdomLab/.github \
--json headRefOid,baseRefOid,reviewDecision,mergeStateStatus,statusCheckRollup \
--jq '{headRefOid,baseRefOid,reviewDecision,mergeStateStatus,checks:[.statusCheckRollup[]|{name,conclusion,status,headSha}]}'Length of output: 109140
I verified the following:
I cannot approve this SHA yet. GitHub currently reports
|
1 participant
Problem
The central OpenCode changed-source coverage gate treated every changed JavaScript-family file as application runtime unless it matched a small exact-name list. Inkspan PR #60 therefore passed 492 tests with 100% statements, branches, functions, and lines for product runtime, but central evidence still failed because Vitest correctly did not instrument:
vite.autosave.config.tsscripts/verify-framework-free-autosave-package.mjsscripts/verify-package.mjsThis was a classifier mismatch, not uncovered product behavior.
Change
The classifier excludes only two bounded non-product categories:
.config.<js-family extension>; andcheck-*orverify-*commands in ascriptsdirectory that is not belowsrc.The gate remains fail-closed for product code:
src/feature.config.tsremains runtime;scripts/serve-package.mjsremains runtime;src/scripts/verify-session.tsremains runtime;coverage-final.jsonstill fails; andThe rule is repository-relative and package-neutral, so it applies consistently to standalone repositories, nested modules, Inkspan, naruon, and other CWL MSA consumers without PR-specific exceptions.
Hard-gate baseline alignment
The first exact-head Python Security run correctly failed the protected-base Strix lock because it still pinned vulnerable
aiohttp==3.14.1andcryptography==49.0.0. This branch carries the byte-identical reviewed input and generated hash locks from exact green head7616fd80d0a0c6cc6e1ff1544728241dbc0ea985of #743:aiohttp==3.14.3cryptography==50.0.0The protected-base CodeQL workflow contract also exposed mixed CodeQL Action revisions during exact-head coverage evidence. Repair commit
9d43004cbbaa91a8cf477a71e64e1a52176a00adaligns init, analyze, and upload-sarif to immutable v4.37.5 commitd1ba80a13dd99fba24a470575428917156a28b43without weakening the coverage classifier or security threshold.TDD and exact-head verification
Current exact head:
33d6168de6aef22f8a76dd19fcc88d99f3613d9e.Focused verification on the implementation commits proves:
scripts/ci/javascript_coverage_gate.py: 267/267 statements and 124/124 branches;The current head adds only the exact-head control-plane failure diagnosis to the authoritative doctoring record. All current-head CI, security, OpenCode, Noema, independent approval, and branch-protection evidence must rerun and succeed before merge. Prior-head success, approval, or requested-changes evidence is not transferable; queued or absent evidence is not treated as success.
Standards and doctoring
docs/doctoring/javascript-runtime-coverage-scope.mdrecords the root cause, fail-closed boundaries, modular behavior, verification contract, the exact-head CodeQL consistency repair, and APA 7 references to current Vitest coverage/configuration documentation, NIST SSDF 1.1, and the SSDF 1.2 initial public draft. NIST still lists SSDF 1.2 as a draft as of August 5, 2026; no formal conformity is claimed.Dependency order
Merge this central control-plane repair through normal protection before rerunning OpenCode review on immutable Inkspan PR #60 head
b02f1662c7118faf1aaf44b8ea1efb376cfc3a85.