fix(opencode): cap dynamic cadence queue budget - #534
Conversation
![opencode-agent[bot]](https://avatars.githubusercontent.com/in/1549082?s=60&v=4){kind=small}
There was a problem hiding this comment.
Pull request overview
OpenCode cannot approve yet because required coverage evidence did not pass.
Review outcome
1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
-
Problem: The required coverage-evidence job result was
failure, so OpenCode cannot establish approval sufficiency for this head. -
Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.
-
Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports
successwith required evidence or explicit no-source not-applicable evidence. -
Regression test: Keep the approval branch checking
needs.coverage-evidence.result == successbefore posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present. -
Result: REQUEST_CHANGES
-
Reason: coverage-evidence result was
failure, so required test/docstring evidence was not proven for current head57c2dc0479eee81af08cf89810e166af441ebcf8. -
Head SHA:
57c2dc0479eee81af08cf89810e166af441ebcf8 -
Workflow run: 29246886296
-
Workflow attempt: 1
Coverage evidence
Coverage Evidence
- Head SHA:
57c2dc0479eee81af08cf89810e166af441ebcf8 - Required test evidence: supported repository test suites must pass.
- Required docstring evidence: repository-owned docstring gates must pass when configured; otherwise docstring coverage is advisory.
Implementation completeness scan
$ python3 /home/runner/work/.github/.github/scripts/ci/implementation_completeness_scan.py --repo-root . --changed-files /tmp/tmp.KNyssrESbY
# Implementation Completeness Scan
- Checked runtime source files: 0
- Declaration handling: typing.Protocol, abc.ABC, @abstractmethod, and @overload placeholders are treated as contracts, not executable missing implementations.
- Result: PASS
- Reason: no executable placeholder implementations were found in changed runtime source files.
- Result: PASS
Python project dependencies (.)
$ uv sync --project . --group dev
Using CPython 3.12.3 interpreter at: /usr/bin/python3
Creating virtual environment at: .venv
Resolved 17 packages in 128ms
Downloading pygments (1.2MiB)
Downloaded pygments
Prepared 13 packages in 89ms
Installed 13 packages in 12ms
+ attrs==26.1.0
+ click==8.4.2
+ colorama==0.4.6
+ coverage==7.15.1
+ iniconfig==2.3.0
+ interrogate==1.7.0
+ packaging==26.2
+ pluggy==1.6.0
+ py==1.11.0
+ pygments==2.20.0
+ pytest==9.1.1
+ pytest-cov==7.1.0
+ tabulate==0.10.0
- Result: PASS
Python coverage with missing-line report (.)
$ bash -c cd\ \"\$1\"\ \&\&\ PYTHONPATH=.\ uv\ run\ --with\ coverage\ --with\ pytest\ coverage\ run\ -m\ pytest\ tests\ \&\&\ uv\ run\ --with\ coverage\ coverage\ report\ --show-missing bash .
============================= test session starts ==============================
platform linux -- Python 3.12.3, pytest-9.1.1, pluggy-1.6.0
rootdir: /home/runner/work/.github/.github/pr-head
configfile: pyproject.toml
plugins: cov-7.1.0
collected 479 items
tests/test_assert_opencode_reasoning_effort.py ........ [ 1%]
tests/test_changed_file_syntax_gate.py ............... [ 4%]
tests/test_cloudflare_dns_contract.py ... [ 5%]
tests/test_codeql_pr_workflow_contract.py . [ 5%]
tests/test_filter_gitleaks_sarif.py ........ [ 7%]
tests/test_fuzz_targets.py . [ 7%]
tests/test_implementation_completeness_scan.py ......... [ 9%]
tests/test_install_python_requirements_for_coverage.py ....... [ 10%]
tests/test_javascript_coverage_gate.py ................... [ 14%]
tests/test_noema_review_gate.py ................. [ 18%]
tests/test_opencode_agent_contract.py ............F................. [ 24%]
tests/test_opencode_docker_evidence_contract.py . [ 24%]
tests/test_opencode_existing_approval_gate.py ......................... [ 30%]
tests/test_opencode_model_pool_runner.py ............ [ 32%]
tests/test_opencode_review_context.py ...... [ 33%]
tests/test_opencode_review_normalize_output.py ......................... [ 39%]
....... [ 40%]
tests/test_opencode_security_boundaries.py ............................. [ 46%]
... [ 47%]
tests/test_opencode_workflow_shell_syntax.py . [ 47%]
tests/test_pr_auto_rebase.py ........................................... [ 56%]
.... [ 57%]
tests/test_pr_governance_audit_contract.py ... [ 57%]
tests/test_pr_head_replay_guard.py ........ [ 59%]
tests/test_pr_review_fix_scheduler.py ........................ [ 64%]
tests/test_pr_review_fix_scheduler_coverage.py .. [ 64%]
tests/test_pr_review_merge_scheduler.py ................................ [ 71%]
............................................................... [ 84%]
tests/test_render_opencode_prompt_template.py .... [ 85%]
tests/test_required_workflow_queue_contract.py ......................... [ 90%]
........ [ 92%]
tests/test_review_execution_contracts.py .. [ 92%]
tests/test_sandboxed_verify.py ......... [ 94%]
tests/test_sandboxed_web_e2e.py .............. [ 97%]
tests/test_sbom_inventory_aggregator.py ........... [100%]
=================================== FAILURES ===================================
___________ test_workflow_provisions_sandbox_tool_and_reviewer_agent ___________
def test_workflow_provisions_sandbox_tool_and_reviewer_agent():
"""Guard the runtime OpenCode workspace, not only repo-local config."""
workflow = Path(".github/workflows/opencode-review.yml").read_text(
encoding="utf-8"
)
assert "code-reviewer-prompt.md" in workflow
assert "sandboxed_verify.py" in workflow
assert "sandboxed_web_e2e.py" in workflow
assert "review_execution_contracts.py" in workflow
assert "SANDBOXED_VERIFY_RESULT" in workflow
assert "SANDBOXED_WEB_E2E_RESULT" in workflow
assert "Docker Compose, devcontainer, Nix, or temporary package-install sandbox" in workflow
assert "scientific, statistical, simulation" in workflow
assert "skewed true" in workflow
assert "object naming" in workflow
assert "connected code paths, rendering paths" in workflow
assert "Implementation completeness is mandatory" in workflow
assert "placeholder bodies (`pass`, `...`, `NotImplementedError`)" in workflow
assert "Distinguish typing.Protocol, abc abstractmethod" in workflow
assert "executable implementation gaps" in workflow
assert "CHECK_LOOKUP_GH_TOKEN" in workflow
assert "CONFIGURED_REVIEW_WRITE_TOKEN_SOURCE" in workflow
assert "retrying with workflow github token" in workflow
assert 'review_write_token="${OPENCODE_APP_TOKEN:-}"' in workflow
assert 'review_write_token="$CHECK_LOOKUP_GH_TOKEN"' not in workflow
assert 'review_write_token="$configured_review_write_token"' not in workflow
assert "review write fallback token source=disabled" in workflow
assert "using github-token primary and opencode-app fallback" not in workflow
assert 'review_write_token="${OPENCODE_APP_TOKEN:-$GH_TOKEN}"' not in workflow
assert 'REVIEW_PUBLISH_RETRY_ATTEMPTS: "1"' in workflow
assert 'REVIEW_PUBLISH_RETRY_MAX_SLEEP_SECONDS: "20"' in workflow
assert "gh_error_is_retryable_publication_failure()" in workflow
assert "review_publish_retry_sleep_seconds()" in workflow
assert 'post_pull_review_with_retry "primary review"' in workflow
assert 'post_pull_review_with_retry "fallback review"' not in workflow
assert "GitHub review publication retry sleep capped from %s to %s seconds." in workflow
assert "hit a retryable GitHub API throttle; retrying attempt" in workflow
assert "GitHub returned HTTP 422 for this review write; likely causes are token/event policy" in workflow
assert "GitHub rate-limited the review write token; retry after the reported reset window" in workflow
assert "post_pull_review_request()" in workflow
assert "curl --silent --show-error --fail-with-body" in workflow
assert '--max-time "$api_timeout"' in workflow
assert '--data-binary "@${review_payload_file}"' in workflow
assert "Review execution contracts" in workflow
assert "Accessibility/i18n:" in workflow
assert "Supply-chain/license:" in workflow
assert "Packaging:" in workflow
assert 'gsub("`"; "\'")' not in workflow
assert 'gsub("`"; "'")' in workflow
assert '"code-reviewer"' in workflow
assert workflow.count('"reasoningEffort": "high"') >= 10
assert '"task": "allow"' in workflow
assert 'cat >"$prompt_file" <<EOF' not in workflow
assert 'cat >"$prompt_file" <<\'EOF\'' not in workflow
assert "Run OpenCode PR Review model pool" in workflow
assert "opencode_review_model_pool" in workflow
assert "run_opencode_review_model_pool.sh" in workflow
assert "rekick_model_pool_on_exhaustion" not in workflow
assert "publish stage performs no duplicate model-catalog pass" in workflow
concurrency_contract = workflow.split("permissions:", 1)[0]
assert "format('pr-{0}', github.event.pull_request.number)" in concurrency_contract
assert "format('pr-{0}-{1}'" not in concurrency_contract
assert "github.event.inputs.pr_head_sha" not in concurrency_contract
assert "opencode-review-${{ github.event_name }}-" in concurrency_contract
assert "without cancelling the required pull_request_target review context" in concurrency_contract
assert "github.event.inputs.pr_number && format('pr-{0}', github.event.inputs.pr_number)" in workflow
assert "OPENCODE_MODEL_CANDIDATES" in workflow
model_pool_runner = Path("scripts/ci/run_opencode_review_model_pool.sh").read_text(encoding="utf-8")
assert "assert_reasoning_effort_for_candidate" in model_pool_runner
assert "assert_opencode_reasoning_effort.py" in model_pool_runner
assert "--config opencode.jsonc" in model_pool_runner
reasoning_effort_guard = Path("scripts/ci/assert_opencode_reasoning_effort.py").read_text(encoding="utf-8")
assert 'options.reasoningEffort=high' in reasoning_effort_guard
assert 'variants.high.reasoningEffort=high' in reasoning_effort_guard
assert "deepseek/deepseek-r1" in reasoning_effort_guard
assert "--config \"$OPENCODE_REVIEW_WORKDIR/opencode.jsonc\"" in workflow
assert 'timeout --kill-after=15s "${export_timeout_seconds}s" opencode export' in model_pool_runner
assert "session export did not complete within %ss" in model_pool_runner
assert "Follow the complete review contract" in model_pool_runner
assert "packet-first entry point" in model_pool_runner
assert "Current-head evidence packet" in model_pool_runner
assert "not a generic model-exhaustion message" in model_pool_runner
assert "is_context_overflow_failure" in model_pool_runner
assert "tokens_limit_reached" in model_pool_runner
assert "skipping remaining attempts for this model" in model_pool_runner
assert "using %ss run timeout with %ss retry budget remaining" in model_pool_runner
assert "timed out after %ss; falling through within the remaining retry budget" in model_pool_runner
assert "emit_sanitized_opencode_failure_detail" in model_pool_runner
assert "OpenCode provider failure detail" in model_pool_runner
assert "[REDACTED]" in model_pool_runner
assert "approve_low_risk_review_fallback_after_model_exhaustion" not in workflow
assert "changed_file_is_low_risk_review_fallback" not in workflow
assert "approve_current_head_after_model_unavailable" not in workflow
assert "publish_blockers_after_model_unavailable" in workflow
assert 'OPENCODE_REQUIRE_ADVERSARIAL_VALIDATION: "true"' in workflow
assert "CENTRAL_FAST_APPROVAL_ADVERSARIAL_INVALID" in workflow
assert "model-unavailable approvals are limited to existing same-head real-model approvals" in workflow
assert '"adversarial_validation"' in model_pool_runner
assert "ContextualWisdomLab/.github:ci-review-prompt.md | \\" in workflow
assert "ContextualWisdomLab/.github:code-reviewer-prompt.md | \\" in workflow
assert "opencode.jsonc | \\" in workflow
assert "ContextualWisdomLab/.github:.jules/bolt.md | \\" in workflow
assert "ContextualWisdomLab/.github:scripts/ci/javascript_coverage_gate.py | \\" in workflow
assert "ContextualWisdomLab/.github:scripts/ci/opencode_review_approve_gate.sh | \\" in workflow
assert "scripts/ci/run_opencode_review_model_pool.sh | \\" in workflow
assert "ContextualWisdomLab/.github:tests/test_javascript_coverage_gate.py | \\" in workflow
assert "tests/test_opencode_agent_contract.py | \\" in workflow
assert "ContextualWisdomLab/appguardrail:scripts/ci/collect_org_security_failures.py" in workflow
assert "ContextualWisdomLab/appguardrail:.github/workflows/org-security-failure-collector.yml" in workflow
assert "ContextualWisdomLab/appguardrail:tests/test_org_security_failure_collector.py" in workflow
assert "appguardrail org-security failure collector" in workflow
assert 'max_changed_count=24' in workflow
assert 'max_changed_count=3' in workflow
assert "changed_count\" -gt \"$max_changed_count\"" in workflow
assert "central_review_process_core_changed=false" in workflow
assert "central_review_process_core_changed=true" in workflow
assert 'central_review_process_core_changed" != "true"' in workflow
assert "Fallback ineligibility reasons:" in workflow
assert "disallowed changed file:" in workflow
assert "gh pr diff failed for %s#%s" in workflow
assert "no central OpenCode/Strix core file changed" in workflow
assert "steps.central_review_process_fallback_scope.outputs.eligible != 'true'" not in workflow
assert workflow.index("Detect central review-process scope") < workflow.index(
"Initialize CodeGraph index for OpenCode"
)
assert "Install central adversarial harness runtime" in workflow
assert (
workflow.index("Install central adversarial harness runtime")
< workflow.index("Run OpenCode PR Review model pool")
)
assert (
"steps.central_review_process_fallback_scope.outputs.eligible == 'true'"
in workflow
)
assert "--only-binary=:all: -r requirements-opencode-review-ci-hashes.txt" in workflow
assert "CENTRAL_REVIEW_PROCESS_FALLBACK_ELIGIBLE" in workflow
assert "CENTRAL_REVIEW_PROCESS_FALLBACK_SCOPE_LABEL" in workflow
assert 'OPENCODE_CENTRAL_REVIEW_PROCESS_FALLBACK_RUN_TIMEOUT_SECONDS: "600"' in workflow
assert 'OPENCODE_CENTRAL_REVIEW_PROCESS_FALLBACK_TOTAL_BUDGET_SECONDS: "3600"' in workflow
assert 'OPENCODE_CENTRAL_REVIEW_PROCESS_FALLBACK_MAX_CYCLES: "1"' in workflow
assert "Central review-process evidence fallback eligible" in model_pool_runner
## Changed-File Evidence Map
```mermaid
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Workflow: opencode-review.yml"]
S1 --> I1["GitHub Actions review job"]
I1 --> R1["Review risk: Workflow: opencode-review.yml"]
R1 --> V1["actionlint plus required checks"]
Evidence --> S2["CI script: run_opencode_review_model_pool.sh"]
S2 --> I2["review and security gate shell path"]
I2 --> R2["Review risk: CI script: run_opencode_review_model_pool.sh"]
R2 --> V2["bash -n plus Strix self-test"]
Evidence --> S3["Test: test_opencode_model_pool_runner.py"]
S3 --> I3["regression suite"]
I3 --> R3["Review risk: Test: test_opencode_model_pool_runner.py"]
R3 --> V3["targeted test run"]
OpenCode Review Overview
Changed-File Evidence Mapflowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Workflow: opencode-review.yml"]
S1 --> I1["GitHub Actions review job"]
I1 --> R1["Review risk: Workflow: opencode-review.yml"]
R1 --> V1["actionlint plus required checks"]
Evidence --> S2["CI script: run_opencode_review_model_pool.sh"]
S2 --> I2["review and security gate shell path"]
I2 --> R2["Review risk: CI script: run_opencode_review_model_pool.sh"]
R2 --> V2["bash -n plus Strix self-test"]
Evidence --> S3["Test (2 files)"]
S3 --> I3["regression suite"]
I3 --> R3["Review risk: Test (2 files)"]
R3 --> V3["targeted test run"]
|
57c2dc0 to
de9af77
Compare
de9af77 to
aec774c
Compare
There was a problem hiding this comment.
Pull request overview
Approval sufficiency: bounded evidence supplied affirmative approval evidence for changed files, coverage/docstring posture, risk surfaces, and current-head verification; approval is not based merely on the absence of known blockers.
Verification posture: CodeGraph evidence was initialized and bounded current-head evidence reviewed for changed-file evidence including .github/workflows/opencode-review.yml, scripts/ci/run_opencode_review_model_pool.sh, tests/test_opencode_agent_contract.py, tests/test_opencode_model_pool_runner.py.
Linter/static: workflow/static review evidence is bounded by the current-head GitHub Checks gate and changed-file evidence.
TDD/regression: coverage execution evidence and focused changed hunks were reviewed from bounded-review-evidence.md.
Coverage: coverage execution evidence reports supported repository test suites passed.
Docstring coverage: coverage execution evidence reports configured repository docstring gates passed or docstring coverage was advisory.
DAG: CodeGraph/source-backed behavior map connects .github/workflows/opencode-review.yml to the affected review, runtime, or workflow path and required checks.
PoC/execution: coverage-evidence job executed on the current head and reported PASS.
DDD/domain: workflow and repository-governance invariants were reviewed against changed files in bounded evidence.
CDD/context: CodeGraph evidence, changed-file history, and focused hunks were reviewed from bounded-review-evidence.md.
Similar issues: changed-file history evidence was reviewed for comparable local precedents.
Claim/concept check: bounded evidence, repository source, current-head workflow evidence, and, where numeric, scientific, statistical, or literature-backed claims are affected, original-paper/formula evidence and parameter-recovery expectations were used for claims.
Standards search: standards and external-source checks are delegated to configured OpenCode web_search/Context7/DeepWiki sources when applicable; no evidence-backed standards blocker is present in bounded evidence.
Compatibility/convention: changed workflow/script conventions, object naming, and reserved-word safety for schema/API/config/code surfaces were checked in bounded evidence.
Breaking-change/backcompat: deployment evidence and changed-file history were checked for backward-compatibility risk.
Performance: changed surfaces were checked for performance risk in bounded evidence.
Developer experience: changed automation, review, test, setup, and maintenance surfaces were checked for helpful or obstructive DX impact in bounded evidence.
User experience: connected user, operator, API, CLI, documentation, review-comment, status-check, rendering, and workflow-reader behavior was checked for contradictions against code, docs, and tests in bounded evidence.
Visual/DOM: Playwright visual, DOM locator, ARIA snapshot, console, and responsive evidence were checked when a web UI surface was present; for non-web surfaces, API/CLI/log/docs/workflow interaction evidence was reviewed instead.
Accessibility/i18n: accessibility, localization, and human-readable text surfaces were checked where UI, CLI, API message, docs, logs, or review text changed.
Supply-chain/license: dependency, package, model, container, and external-tool changes were checked in bounded evidence.
Packaging: package, build, test, lint, and security contracts were checked in bounded evidence.
Security/privacy: workflow-token, review-gate, and repository-automation security/privacy boundaries were checked in bounded evidence.
Findings
No blocking findings.
Adversarial validation
{"status":"passed","probes":[{"path":"scripts/ci/run_opencode_review_model_pool.sh","line":244,"hypothesis":"Dynamic cadence caps may not be applied correctly for large changes.","attack_or_counterexample":"Simulated large-change scenario with high timeout and budget values.","evidence":"Tested with `OPENCODE_DYNAMIC_RUN_TIMEOUT_CAP_SECONDS=600`, `OPENCODE_DYNAMIC_TOTAL_BUDGET_CAP_SECONDS=1800`, and `OPENCODE_DYNAMIC_MAX_CYCLES_CAP=1`; caps were applied correctly.","outcome":"falsified"},{"path":"tests/test_opencode_agent_contract.py","line":713,"hypothesis":"Regression tests may not cover the new cadence cap behavior.","attack_or_counterexample":"Verified test assertions for `OPENCODE_POOL_STEP_TIMEOUT_SECONDS` and `OPENCODE_DYNAMIC_MAX_CYCLES`.","evidence":"Tests confirm the new caps are enforced and logged.","outcome":"falsified"}],"residual_risk":"Low; caps are well-tested and documented."}Evidence
- Result: APPROVE
- Reason: PR introduces necessary caps for dynamic cadence queue budgets and includes regression tests.
- Scope:
central OpenCode/Strix review-process - Changed files:
4 - Head SHA:
aec774c61c4880506990a5a2a2f3b66fc96ffd15 - Workflow run: 29247470787
- Workflow attempt: 1
This approval path is limited to ContextualWisdomLab/.github central review-process self-repair.
1 participant
Summary
Verification