feat: unify Noema, OpenCode, and Strix free-first fallbacks - #760
feat: unify Noema, OpenCode, and Strix free-first fallbacks#760seonghobae wants to merge 12 commits into
Conversation
|
Warning Review limit reached
Next review available in: 2 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (28)
Comment |
63b5c1d to
6c99cbc
Compare
|
@coderabbitai review @opencode-agent @cwl-noema-review exact-current-head review requested for |
|
I will verify the requested policy, security, boundary, and evidence requirements. I will approve only if the exact-head review and required checks succeed.
|
![opencode-agent[bot]](https://avatars.githubusercontent.com/in/1549082?s=60&v=4){kind=small}
There was a problem hiding this comment.
Pull request overview
OpenCode cannot approve yet because required coverage evidence did not pass.
Review outcome
1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
-
Problem: The required coverage-evidence job result was
failure, so OpenCode cannot establish approval sufficiency for this head. -
Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.
-
Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports
successwith required evidence or explicit no-source not-applicable evidence. -
Regression test: Keep the approval branch checking
needs.coverage-evidence.result == successbefore posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present. -
Result: REQUEST_CHANGES
-
Reason: coverage-evidence result was
failure, so required test/docstring evidence was not proven for current head6c99cbc25bee6e94b61e7fbdad59afd9893e56f9. -
Head SHA:
6c99cbc25bee6e94b61e7fbdad59afd9893e56f9 -
Workflow run: 30961482014
-
Workflow attempt: 1
Coverage evidence
Coverage Decision
- Result: FAIL
- Test evidence: not proven passing
- Docstring evidence: not proven passing when configured
- Failure count: 1
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Changed file (9 files)"]
S1 --> I1["repository behavior"]
I1 --> R1["Review risk: Changed file (9 files)"]
R1 --> V1["required checks"]
Evidence --> S2["Docs (2 files)"]
S2 --> I2["operator or user guidance"]
I2 --> R2["Review risk: Docs (2 files)"]
R2 --> V2["docs review"]
Evidence --> S3["CI script (6 files)"]
S3 --> I3["review and security gate shell path"]
I3 --> R3["Review risk: CI script (6 files)"]
R3 --> V3["bash -n plus Strix self-test"]
Evidence --> S4["Test (7 files)"]
S4 --> I4["regression suite"]
I4 --> R4["Review risk: Test (7 files)"]
R4 --> V4["targeted test run"]
OpenCode Review Overview
Pull request overviewOpenCode cannot approve yet because required coverage evidence did not pass. Review outcome1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
Coverage evidenceCoverage Decision
Changed-File Evidence Mapflowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Changed file (11 files)"]
S1 --> I1["repository behavior"]
I1 --> R1["Review risk: Changed file (11 files)"]
R1 --> V1["required checks"]
Evidence --> S2["Docs (2 files)"]
S2 --> I2["operator or user guidance"]
I2 --> R2["Review risk: Docs (2 files)"]
R2 --> V2["docs review"]
Evidence --> S3["CI script (6 files)"]
S3 --> I3["review and security gate shell path"]
I3 --> R3["Review risk: CI script (6 files)"]
R3 --> V3["bash -n plus Strix self-test"]
Evidence --> S4["Test (9 files)"]
S4 --> I4["regression suite"]
I4 --> R4["Review risk: Test (9 files)"]
R4 --> V4["targeted test run"]
|
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
|
Exact-head security repair at
The previous dependency-review and Semgrep failures were therefore addressed without excluding either file, weakening the scanners, changing reviewer credentials, or changing the free-first execution policy. Fresh exact-head checks remain authoritative, and cross-repository order #96 → #94 → #760 is unchanged. |
|
@opencode-agent @cwl-noema-review Please independently review exact current head |
There was a problem hiding this comment.
Pull request overview
OpenCode cannot approve yet because required coverage evidence did not pass.
Review outcome
1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
-
Problem: The required coverage-evidence job result was
failure, so OpenCode cannot establish approval sufficiency for this head. -
Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.
-
Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports
successwith required evidence or explicit no-source not-applicable evidence. -
Regression test: Keep the approval branch checking
needs.coverage-evidence.result == successbefore posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present. -
Result: REQUEST_CHANGES
-
Reason: coverage-evidence result was
failure, so required test/docstring evidence was not proven for current head49aacff9e1a4907ae9eae4e64fad5881968a7053. -
Head SHA:
49aacff9e1a4907ae9eae4e64fad5881968a7053 -
Workflow run: 30971144855
-
Workflow attempt: 1
Coverage evidence
Coverage Decision
- Result: FAIL
- Test evidence: not proven passing
- Docstring evidence: not proven passing when configured
- Failure count: 1
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Changed file (11 files)"]
S1 --> I1["repository behavior"]
I1 --> R1["Review risk: Changed file (11 files)"]
R1 --> V1["required checks"]
Evidence --> S2["Docs (2 files)"]
S2 --> I2["operator or user guidance"]
I2 --> R2["Review risk: Docs (2 files)"]
R2 --> V2["docs review"]
Evidence --> S3["CI script (6 files)"]
S3 --> I3["review and security gate shell path"]
I3 --> R3["Review risk: CI script (6 files)"]
R3 --> V3["bash -n plus Strix self-test"]
Evidence --> S4["Test (9 files)"]
S4 --> I4["regression suite"]
I4 --> R4["Review risk: Test (9 files)"]
R4 --> V4["targeted test run"]
2 participants
Summary
Unifies the model-selection policy used by Noema, OpenCode Agent, and Strix without replacing their existing provider transports, reviewer identities, credential scopes, result validators, or security gates.
contextual-orchestratorfallback-policy source through a receipt-verified vendored module;Agent behavior
Noema
Public repositories try three approved NVIDIA NIM hosted models first. An explicitly configured custom provider remains the final fallback. Private/internal repositories cannot use public-only trial candidates.
OpenCode Agent
The existing configured pool is intersected with the shared manifest and reordered: NVIDIA NIM free models →
opencode-free/*→ included-quota GitHub Models → paid OpenCode/OpenAI/OpenRouter candidates. The unchanged core still owns retries, output normalization, evidence sealing, secret masking, and fail-closed exhaustion.Strix
The shared plan is applied before the unchanged Strix scan gate resolves its primary model. The gate still selects provider-specific keys/API bases, parses security findings, enforces severity thresholds, and fails closed on provider warning or timeout signals.
Verification
Local exact-slice verification established:
74 passed487 statements,180 branches,100%contextual_fallback_policy.py:174 statements,56 branches,100%93 statements,36 branches,100%100%statement and branch coveragecompileallpassedA source-pin regression additionally requires the adapter and
VENDOR_RECEIPT.jsonto identify integrated upstream commit40c6a4b419cdf8fa90c422acb5443a0e1cca5d16. Repository-wide exact-head checks remain authoritative.Cross-repository dependency
ContextualWisdomLab/contextual-orchestrator#96combines the provider-egress hardening and interpreter-portable Atheris prerequisite. Stacked PRContextualWisdomLab/contextual-orchestrator#94adds the transport-neutral free-first policy on that integrated base, producing immutable source commit40c6a4b419cdf8fa90c422acb5443a0e1cca5d16pinned here.This central PR must not merge until #96 and then #94 have passed exact-head checks and merged without bypass. The receipt verifies the integrated source commit, exact file maps, regular-file status, and unchanged fallback-policy Git blob identities, preventing mutable-branch or ancestor-only substitution.
Documentation
docs/shared-llm-fallback-policy.mddocs/doctoring/shared-llm-fallback-policy.mdwith APA 7 referencesCHANGELOG.mdVerification gate
Merge only after the cross-repository dependency order completes and exact-head central tests, 100% statement/branch coverage, complete docstrings, Secret Scan, Semgrep, CodeQL, Python Security, supply-chain checks, Noema, OpenCode, Strix, CodeRabbit, independent current-head approval, and branch protection pass. No reviewer credential name or scope changes.