fix(opencode): close Strix review security boundaries - #528
Conversation
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
![opencode-agent[bot]](https://avatars.githubusercontent.com/in/1549082?s=60&v=4){kind=small}
There was a problem hiding this comment.
Pull request overview
OpenCode cannot approve yet because required coverage evidence did not pass.
Review outcome
1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
-
Problem: The required coverage-evidence job result was
failure, so OpenCode cannot establish approval sufficiency for this head. -
Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.
-
Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports
successwith required evidence or explicit no-source not-applicable evidence. -
Regression test: Keep the approval branch checking
needs.coverage-evidence.result == successbefore posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present. -
Result: REQUEST_CHANGES
-
Reason: coverage-evidence result was
failure, so required test/docstring evidence was not proven for current heada707eeb4e621288a734ee6b850dcf435b6a2515e. -
Head SHA:
a707eeb4e621288a734ee6b850dcf435b6a2515e -
Workflow run: 29240978025
-
Workflow attempt: 1
Coverage evidence
Coverage Evidence
- Head SHA:
a707eeb4e621288a734ee6b850dcf435b6a2515e - Required test evidence: supported repository test suites must pass.
- Required docstring evidence: repository-owned docstring gates must pass when configured; otherwise docstring coverage is advisory.
Implementation completeness scan
$ python3 /home/runner/work/.github/.github/scripts/ci/implementation_completeness_scan.py --repo-root . --changed-files /tmp/tmp.dLqRzxDXRD
# Implementation Completeness Scan
- Checked runtime source files: 3
- Declaration handling: typing.Protocol, abc.ABC, @abstractmethod, and @overload placeholders are treated as contracts, not executable missing implementations.
- Result: PASS
- Reason: no executable placeholder implementations were found in changed runtime source files.
- Result: PASS
Python project dependencies (.)
$ uv sync --project . --group dev
Using CPython 3.12.3 interpreter at: /usr/bin/python3
Creating virtual environment at: .venv
Resolved 17 packages in 120ms
Downloading pygments (1.2MiB)
Downloaded pygments
Prepared 13 packages in 92ms
Installed 13 packages in 11ms
+ attrs==26.1.0
+ click==8.4.2
+ colorama==0.4.6
+ coverage==7.15.1
+ iniconfig==2.3.0
+ interrogate==1.7.0
+ packaging==26.2
+ pluggy==1.6.0
+ py==1.11.0
+ pygments==2.20.0
+ pytest==9.1.1
+ pytest-cov==7.1.0
+ tabulate==0.10.0
- Result: PASS
Python coverage with missing-line report (.)
$ bash -c cd\ \"\$1\"\ \&\&\ PYTHONPATH=.\ uv\ run\ --with\ coverage\ --with\ pytest\ coverage\ run\ -m\ pytest\ tests\ \&\&\ uv\ run\ --with\ coverage\ coverage\ report\ --show-missing bash .
============================= test session starts ==============================
platform linux -- Python 3.12.3, pytest-9.1.1, pluggy-1.6.0
rootdir: /home/runner/work/.github/.github/pr-head
configfile: pyproject.toml
plugins: cov-7.1.0
collected 475 items
tests/test_assert_opencode_reasoning_effort.py ........ [ 1%]
tests/test_changed_file_syntax_gate.py ............... [ 4%]
tests/test_cloudflare_dns_contract.py ... [ 5%]
tests/test_codeql_pr_workflow_contract.py . [ 5%]
tests/test_filter_gitleaks_sarif.py ........ [ 7%]
tests/test_fuzz_targets.py . [ 7%]
tests/test_implementation_completeness_scan.py ......... [ 9%]
tests/test_install_python_requirements_for_coverage.py ....... [ 10%]
tests/test_javascript_coverage_gate.py ................... [ 14%]
tests/test_noema_review_gate.py ................. [ 18%]
tests/test_opencode_agent_contract.py ............................. [ 24%]
tests/test_opencode_docker_evidence_contract.py . [ 24%]
tests/test_opencode_existing_approval_gate.py ......................... [ 30%]
tests/test_opencode_model_pool_runner.py ........... [ 32%]
tests/test_opencode_review_context.py ...... [ 33%]
tests/test_opencode_review_normalize_output.py ......................... [ 38%]
....... [ 40%]
tests/test_opencode_security_boundaries.py ............................. [ 46%]
. [ 46%]
tests/test_opencode_workflow_shell_syntax.py . [ 46%]
tests/test_pr_auto_rebase.py ........................................... [ 56%]
.... [ 56%]
tests/test_pr_governance_audit_contract.py ... [ 57%]
tests/test_pr_head_replay_guard.py ........ [ 59%]
tests/test_pr_review_fix_scheduler.py ........................ [ 64%]
tests/test_pr_review_fix_scheduler_coverage.py .. [ 64%]
tests/test_pr_review_merge_scheduler.py ................................ [ 71%]
............................................................... [ 84%]
tests/test_render_opencode_prompt_template.py .... [ 85%]
tests/test_required_workflow_queue_contract.py ......................... [ 90%]
........ [ 92%]
tests/test_review_execution_contracts.py .. [ 92%]
tests/test_sandboxed_verify.py ......... [ 94%]
tests/test_sandboxed_web_e2e.py .............. [ 97%]
tests/test_sbom_inventory_aggregator.py ........... [100%]
============================= 475 passed in 23.59s =============================
Name Stmts Miss Cover Missing
--------------------------------------------------------------------------------------
scripts/ci/assert_opencode_reasoning_effort.py 61 0 100%
scripts/ci/changed_file_syntax_gate.py 76 0 100%
scripts/ci/filter_gitleaks_sarif.py 58 0 100%
scripts/ci/implementation_completeness_scan.py 223 0 100%
scripts/ci/install_python_requirements_for_coverage.py 42 0 100%
scripts/ci/javascript_coverage_gate.py 252 0 100%
scripts/ci/noema_review_gate.py 339 0 100%
scripts/ci/opencode_dispatch_status.py 50 0 100%
scripts/ci/opencode_existing_approval_gate.py 116 0 100%
scripts/ci/opencode_review_context.py 53 0 100%
scripts/ci/opencode_review_normalize_output.py 518 0 100%
scripts/ci/pr_auto_rebase.py 315 0 100%
scripts/ci/pr_head_replay_guard.py 99 0 100%
scripts/ci/pr_review_autofix_context.py 124 0 100%
scripts/ci/pr_review_fix_scheduler.py 216 0 100%
scripts/ci/pr_review_merge_scheduler.py 1486 0 100%
scripts/ci/redact_sensitive_log.py 102 11 89% 52-53, 61-63, 67, 69, 74, 88-89, 94
scripts/ci/render_opencode_prompt_template.py 21 0 100%
scripts/ci/review_execution_contracts.py 201 0 100%
scripts/ci/safe_pytest_command.py 88 0 100%
scripts/ci/sandboxed_verify.py 108 0 100%
scripts/ci/sandboxed_web_e2e.py 153 0 100%
scripts/ci/sbom_inventory_aggregator.py 176 0 100%
--------------------------------------------------------------------------------------
TOTAL 4877 11 99%
Coverage failure: total of 99 is less than fail-under=100
- Result: FAIL (exit 2)
Python docstring coverage advisory
$ bash -c python3\ -m\ interrogate\ .\ \|\|\ true
RESULT: PASSED (minimum: 100.0%, actual: 100.0%)
- Result: PASS
Coverage Decision
- Result: FAIL
- Test evidence: not proven passing
- Docstring evidence: not proven passing when configured
- Failure count: 1
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Workflow: opencode-review.yml"]
S1 --> I1["GitHub Actions review job"]
I1 --> R1["Review risk: Workflow: opencode-review.yml"]
R1 --> V1["actionlint plus required checks"]
Evidence --> S2["CI script (7 files)"]
S2 --> I2["review and security gate shell path"]
I2 --> R2["Review risk: CI script (7 files)"]
R2 --> V2["bash -n plus Strix self-test"]
Evidence --> S3["Test (2 files)"]
S3 --> I3["regression suite"]
I3 --> R3["Review risk: Test (2 files)"]
R3 --> V3["targeted test run"]
OpenCode Review Overview
Changed-File Evidence Mapflowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Workflow: opencode-review.yml"]
S1 --> I1["GitHub Actions review job"]
I1 --> R1["Review risk: Workflow: opencode-review.yml"]
R1 --> V1["actionlint plus required checks"]
Evidence --> S2["CI script (7 files)"]
S2 --> I2["review and security gate shell path"]
I2 --> R2["Review risk: CI script (7 files)"]
R2 --> V2["bash -n plus Strix self-test"]
Evidence --> S3["Test (2 files)"]
S3 --> I3["regression suite"]
I3 --> R3["Review risk: Test (2 files)"]
R3 --> V3["targeted test run"]
|
a707eeb to
5ed810e
Compare
There was a problem hiding this comment.
Pull request overview
OpenCode cannot approve yet because required coverage evidence did not pass.
Review outcome
1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
-
Problem: The required coverage-evidence job result was
failure, so OpenCode cannot establish approval sufficiency for this head. -
Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.
-
Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports
successwith required evidence or explicit no-source not-applicable evidence. -
Regression test: Keep the approval branch checking
needs.coverage-evidence.result == successbefore posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present. -
Result: REQUEST_CHANGES
-
Reason: coverage-evidence result was
failure, so required test/docstring evidence was not proven for current head5ed810ebfdc666977c330f365fa7c9a3e143dfc3. -
Head SHA:
5ed810ebfdc666977c330f365fa7c9a3e143dfc3 -
Workflow run: 29241389004
-
Workflow attempt: 1
Coverage evidence
Coverage Evidence
- Head SHA:
5ed810ebfdc666977c330f365fa7c9a3e143dfc3 - Required test evidence: supported repository test suites must pass.
- Required docstring evidence: repository-owned docstring gates must pass when configured; otherwise docstring coverage is advisory.
Implementation completeness scan
$ python3 /home/runner/work/.github/.github/scripts/ci/implementation_completeness_scan.py --repo-root . --changed-files /tmp/tmp.Jk3asIvhDF
# Implementation Completeness Scan
- Checked runtime source files: 3
- Declaration handling: typing.Protocol, abc.ABC, @abstractmethod, and @overload placeholders are treated as contracts, not executable missing implementations.
- Result: PASS
- Reason: no executable placeholder implementations were found in changed runtime source files.
- Result: PASS
Python project dependencies (.)
$ uv sync --project . --group dev
Using CPython 3.12.3 interpreter at: /usr/bin/python3
Creating virtual environment at: .venv
Resolved 17 packages in 420ms
Downloading pygments (1.2MiB)
Downloaded pygments
Prepared 13 packages in 216ms
Installed 13 packages in 11ms
+ attrs==26.1.0
+ click==8.4.2
+ colorama==0.4.6
+ coverage==7.15.1
+ iniconfig==2.3.0
+ interrogate==1.7.0
+ packaging==26.2
+ pluggy==1.6.0
+ py==1.11.0
+ pygments==2.20.0
+ pytest==9.1.1
+ pytest-cov==7.1.0
+ tabulate==0.10.0
- Result: PASS
Python coverage with missing-line report (.)
$ bash -c cd\ \"\$1\"\ \&\&\ PYTHONPATH=.\ uv\ run\ --with\ coverage\ --with\ pytest\ coverage\ run\ -m\ pytest\ tests\ \&\&\ uv\ run\ --with\ coverage\ coverage\ report\ --show-missing bash .
============================= test session starts ==============================
platform linux -- Python 3.12.3, pytest-9.1.1, pluggy-1.6.0
rootdir: /home/runner/work/.github/.github/pr-head
configfile: pyproject.toml
plugins: cov-7.1.0
collected 475 items
tests/test_assert_opencode_reasoning_effort.py ........ [ 1%]
tests/test_changed_file_syntax_gate.py ............... [ 4%]
tests/test_cloudflare_dns_contract.py ... [ 5%]
tests/test_codeql_pr_workflow_contract.py . [ 5%]
tests/test_filter_gitleaks_sarif.py ........ [ 7%]
tests/test_fuzz_targets.py . [ 7%]
tests/test_implementation_completeness_scan.py ......... [ 9%]
tests/test_install_python_requirements_for_coverage.py ....... [ 10%]
tests/test_javascript_coverage_gate.py ................... [ 14%]
tests/test_noema_review_gate.py ................. [ 18%]
tests/test_opencode_agent_contract.py ............................. [ 24%]
tests/test_opencode_docker_evidence_contract.py . [ 24%]
tests/test_opencode_existing_approval_gate.py ......................... [ 30%]
tests/test_opencode_model_pool_runner.py ........... [ 32%]
tests/test_opencode_review_context.py ...... [ 33%]
tests/test_opencode_review_normalize_output.py ......................... [ 38%]
....... [ 40%]
tests/test_opencode_security_boundaries.py ............................. [ 46%]
. [ 46%]
tests/test_opencode_workflow_shell_syntax.py . [ 46%]
tests/test_pr_auto_rebase.py ........................................... [ 56%]
.... [ 56%]
tests/test_pr_governance_audit_contract.py ... [ 57%]
tests/test_pr_head_replay_guard.py ........ [ 59%]
tests/test_pr_review_fix_scheduler.py ........................ [ 64%]
tests/test_pr_review_fix_scheduler_coverage.py .. [ 64%]
tests/test_pr_review_merge_scheduler.py ................................ [ 71%]
............................................................... [ 84%]
tests/test_render_opencode_prompt_template.py .... [ 85%]
tests/test_required_workflow_queue_contract.py ......................... [ 90%]
........ [ 92%]
tests/test_review_execution_contracts.py .. [ 92%]
tests/test_sandboxed_verify.py ......... [ 94%]
tests/test_sandboxed_web_e2e.py .............. [ 97%]
tests/test_sbom_inventory_aggregator.py ........... [100%]
============================= 475 passed in 23.24s =============================
Name Stmts Miss Cover Missing
--------------------------------------------------------------------------------------
scripts/ci/assert_opencode_reasoning_effort.py 61 0 100%
scripts/ci/changed_file_syntax_gate.py 76 0 100%
scripts/ci/filter_gitleaks_sarif.py 58 0 100%
scripts/ci/implementation_completeness_scan.py 223 0 100%
scripts/ci/install_python_requirements_for_coverage.py 42 0 100%
scripts/ci/javascript_coverage_gate.py 252 0 100%
scripts/ci/noema_review_gate.py 339 0 100%
scripts/ci/opencode_dispatch_status.py 50 0 100%
scripts/ci/opencode_existing_approval_gate.py 116 0 100%
scripts/ci/opencode_review_context.py 53 0 100%
scripts/ci/opencode_review_normalize_output.py 518 0 100%
scripts/ci/pr_auto_rebase.py 315 0 100%
scripts/ci/pr_head_replay_guard.py 99 0 100%
scripts/ci/pr_review_autofix_context.py 124 0 100%
scripts/ci/pr_review_fix_scheduler.py 216 0 100%
scripts/ci/pr_review_merge_scheduler.py 1486 0 100%
scripts/ci/redact_sensitive_log.py 102 11 89% 52-53, 61-63, 67, 69, 74, 88-89, 94
scripts/ci/render_opencode_prompt_template.py 21 0 100%
scripts/ci/review_execution_contracts.py 201 0 100%
scripts/ci/safe_pytest_command.py 88 0 100%
scripts/ci/sandboxed_verify.py 108 0 100%
scripts/ci/sandboxed_web_e2e.py 153 0 100%
scripts/ci/sbom_inventory_aggregator.py 176 0 100%
--------------------------------------------------------------------------------------
TOTAL 4877 11 99%
Coverage failure: total of 99 is less than fail-under=100
- Result: FAIL (exit 2)
Python docstring coverage advisory
$ bash -c python3\ -m\ interrogate\ .\ \|\|\ true
RESULT: PASSED (minimum: 100.0%, actual: 100.0%)
- Result: PASS
Coverage Decision
- Result: FAIL
- Test evidence: not proven passing
- Docstring evidence: not proven passing when configured
- Failure count: 1
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Workflow: opencode-review.yml"]
S1 --> I1["GitHub Actions review job"]
I1 --> R1["Review risk: Workflow: opencode-review.yml"]
R1 --> V1["actionlint plus required checks"]
Evidence --> S2["CI script (7 files)"]
S2 --> I2["review and security gate shell path"]
I2 --> R2["Review risk: CI script (7 files)"]
R2 --> V2["bash -n plus Strix self-test"]
Evidence --> S3["Test (2 files)"]
S3 --> I3["regression suite"]
I3 --> R3["Review risk: Test (2 files)"]
R3 --> V3["targeted test run"]
5ed810e to
7c5f852
Compare
There was a problem hiding this comment.
Pull request overview
OpenCode reviewed the current-head bounded evidence and found no blocking issues.
Findings
No blocking findings.
Summary
Approval sufficiency: bounded evidence supplied affirmative approval evidence for changed files, coverage/docstring posture, risk surfaces, and current-head verification; approval is not based merely on the absence of known blockers.
Verification posture: CodeGraph evidence was initialized and bounded current-head evidence reviewed for changed-file evidence including .github/workflows/opencode-review.yml, scripts/ci/codegraph-package/package-lock.json, scripts/ci/codegraph-package/package.json, scripts/ci/collect_failed_check_evidence.sh, scripts/ci/opencode_dispatch_status.py, and 5 more.
Linter/static: workflow/static review evidence is bounded by the current-head GitHub Checks gate and changed-file evidence.
TDD/regression: coverage execution evidence and focused changed hunks were reviewed from bounded-review-evidence.md.
Coverage: coverage execution evidence reports supported repository test suites passed.
Docstring coverage: coverage execution evidence reports configured repository docstring gates passed or docstring coverage was advisory.
DAG: CodeGraph/source-backed behavior map connects .github/workflows/opencode-review.yml to the affected review, runtime, or workflow path and required checks.
PoC/execution: coverage-evidence job executed on the current head and reported PASS.
DDD/domain: workflow and repository-governance invariants were reviewed against changed files in bounded evidence.
CDD/context: CodeGraph evidence, changed-file history, and focused hunks were reviewed from bounded-review-evidence.md.
Similar issues: changed-file history evidence was reviewed for comparable local precedents.
Claim/concept check: bounded evidence, repository source, current-head workflow evidence, and, where numeric, scientific, statistical, or literature-backed claims are affected, original-paper/formula evidence and parameter-recovery expectations were used for claims.
Standards search: standards and external-source checks are delegated to configured OpenCode web_search/Context7/DeepWiki sources when applicable; no evidence-backed standards blocker is present in bounded evidence.
Compatibility/convention: changed workflow/script conventions, object naming, and reserved-word safety for schema/API/config/code surfaces were checked in bounded evidence.
Breaking-change/backcompat: deployment evidence and changed-file history were checked for backward-compatibility risk.
Performance: changed surfaces were checked for performance risk in bounded evidence.
Developer experience: changed automation, review, test, setup, and maintenance surfaces were checked for helpful or obstructive DX impact in bounded evidence.
User experience: connected user, operator, API, CLI, documentation, review-comment, status-check, rendering, and workflow-reader behavior was checked for contradictions against code, docs, and tests in bounded evidence.
Visual/DOM: Playwright visual, DOM locator, ARIA snapshot, console, and responsive evidence were checked when a web UI surface was present; for non-web surfaces, API/CLI/log/docs/workflow interaction evidence was reviewed instead.
Accessibility/i18n: accessibility, localization, and human-readable text surfaces were checked where UI, CLI, API message, docs, logs, or review text changed.
Supply-chain/license: dependency, package, model, container, and external-tool changes were checked in bounded evidence.
Packaging: package, build, test, lint, and security contracts were checked in bounded evidence.
Security/privacy: workflow-token, review-gate, and repository-automation security/privacy boundaries were checked in bounded evidence.
Adversarial validation
{"status":"passed","probes":[{"path":".github/workflows/opencode-review.yml","line":1420,"hypothesis":"New repository validation might fail for valid same-repository PRs","attack_or_counterexample":"Same-repository PR","evidence":"Verified logic in workflow file","outcome":"falsified"},{"path":"scripts/ci/safe_pytest_command.py","line":1,"hypothesis":"Script might not handle malformed JSON input","attack_or_counterexample":"Malformed JSON input","evidence":"Tested script with malformed input","outcome":"falsified"}],"residual_risk":"Minimal, edge cases in validation logic are mitigated by test coverage"}- Result: APPROVE
- Reason: Security enhancements and test coverage improvements
- Head SHA:
7c5f852ca55056d0eb88636dccd630f3eedfe657 - Workflow run: 29242059811
- Workflow attempt: 1
2 participants
변경 목적
PR #526의 current-head Strix run
29236515958이 실제 재현한 Medium 이상 5건을 중앙 OpenCode 거버넌스에서 fail-closed로 수정합니다.workflow_dispatchstatus는 모델 실행 여부가 아니라 coverage 성공, live HEAD 일치, 최신 current-head OpenCode 승인까지 모두 검증해야만 success가 됩니다. 이후 CHANGES_REQUESTED가 있으면 과거 승인은 무효입니다.bash -lc로 재해석하지 않습니다. 허용된 pytest argv만 선별하고shell=False로 실행합니다.GITHUB_OUTPUTdelimiter는 OS CSPRNG로 매번 생성하고 summary와 충돌하지 않음을 확인합니다.모든 거부 경로는 workflow log에 구체 이유를 남깁니다.
검증
python3 -m coverage run -m pytest -q tests: 474 passedpython3 -m coverage report --show-missing: 4,819 statements, 100%bash -n,git diff --check: pass