fix(ci): restore review control plane and respond to agent mentions - #757
fix(ci): restore review control plane and respond to agent mentions#757seonghobae wants to merge 75 commits into
Conversation
|
Warning Review limit reached
Next review available in: 46 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (27)
📝 WalkthroughWalkthroughPR 댓글의 에이전트 멘션 라우팅과 조직 스윕을 추가했습니다. 재사용 워크플로의 불변 소스 검증을 강화했습니다. SBOM·CodeQL 워크플로와 Python 의존성 잠금 및 preflight 검증을 갱신했습니다. Changes에이전트 멘션 자동화
워크플로 무결성 및 보안 기준
Python 의존성 잠금 검증
Estimated code review effort: 5 (Critical) | ~120 minutes Sequence Diagram(s)sequenceDiagram
participant PRCommenter
participant AgentMentionWorkflow
participant MentionRouter
participant CentralDispatch
PRCommenter->>AgentMentionWorkflow: 승인된 PR 댓글 멘션
AgentMentionWorkflow->>MentionRouter: PR 메타데이터와 댓글 전달
MentionRouter->>MentionRouter: 멘션, 권한, 상태, 중복 여부 검증
MentionRouter->>CentralDispatch: Noema 또는 허용된 OpenCode 이벤트 전송
CentralDispatch-->>MentionRouter: 디스패치 결과
MentionRouter->>PRCommenter: eyes 반응과 처리 영수증 게시
Possibly related PRs
Suggested labels: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
![opencode-agent[bot]](https://avatars.githubusercontent.com/in/1549082?s=60&v=4){kind=small}
There was a problem hiding this comment.
Pull request overview
OpenCode cannot approve yet because required coverage evidence did not pass.
Review outcome
1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
-
Problem: The required coverage-evidence job result was
failure, so OpenCode cannot establish approval sufficiency for this head. -
Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.
-
Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports
successwith required evidence or explicit no-source not-applicable evidence. -
Regression test: Keep the approval branch checking
needs.coverage-evidence.result == successbefore posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present. -
Result: REQUEST_CHANGES
-
Reason: coverage-evidence result was
failure, so required test/docstring evidence was not proven for current head0db74c7c99a52d4c1532163d1884f175abf215d1. -
Head SHA:
0db74c7c99a52d4c1532163d1884f175abf215d1 -
Workflow run: 30957028159
-
Workflow attempt: 1
Coverage evidence
Coverage Decision
- Result: FAIL
- Test evidence: not proven passing
- Docstring evidence: not proven passing when configured
- Failure count: 1
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Workflow: agent-mention-router.yml"]
S1 --> I1["GitHub Actions review job"]
I1 --> R1["Review risk: Workflow: agent-mention-router.yml"]
R1 --> V1["actionlint plus required checks"]
Evidence --> S2["Docs: review-agent-comment-invocation.md"]
S2 --> I2["operator or user guidance"]
I2 --> R2["Review risk: Docs: review-agent-comment-invocation.md"]
R2 --> V2["docs review"]
Evidence --> S3["CI script (2 files)"]
S3 --> I3["review and security gate shell path"]
I3 --> R3["Review risk: CI script (2 files)"]
R3 --> V3["bash -n plus Strix self-test"]
Evidence --> S4["Test (3 files)"]
S4 --> I4["regression suite"]
I4 --> R4["Review risk: Test (3 files)"]
R4 --> V4["targeted test run"]
OpenCode Review Overview
Pull request overviewOpenCode cannot approve yet because required coverage evidence did not pass. Review outcome1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
Coverage evidenceCoverage Decision
Changed-File Evidence Mapflowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Workflow (6 files)"]
S1 --> I1["GitHub Actions review job"]
I1 --> R1["Review risk: Workflow (6 files)"]
R1 --> V1["actionlint plus required checks"]
Evidence --> S2["Changed file (3 files)"]
S2 --> I2["repository behavior"]
I2 --> R2["Review risk: Changed file (3 files)"]
R2 --> V2["required checks"]
Evidence --> S3["Docs (5 files)"]
S3 --> I3["operator or user guidance"]
I3 --> R3["Review risk: Docs (5 files)"]
R3 --> V3["docs review"]
Evidence --> S4["CI script (7 files)"]
S4 --> I4["review and security gate shell path"]
I4 --> R4["Review risk: CI script (7 files)"]
R4 --> V4["bash -n plus Strix self-test"]
Evidence --> S5["Test (18 files)"]
S5 --> I5["regression suite"]
I5 --> R5["Review risk: Test (18 files)"]
R5 --> V5["targeted test run"]
|
seonghobae
changed the title
Pull request was converted to draft
|
@coderabbitai review |
✅ Action performedReview finished.
|
|
@coderabbitai review |
✅ Action performedReview finished.
|
|
@coderabbitai review |
There was a problem hiding this comment.
Pull request overview
OpenCode cannot approve yet because required coverage evidence did not pass.
Review outcome
1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
-
Problem: The required coverage-evidence job result was
failure, so OpenCode cannot establish approval sufficiency for this head. -
Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.
-
Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports
successwith required evidence or explicit no-source not-applicable evidence. -
Regression test: Keep the approval branch checking
needs.coverage-evidence.result == successbefore posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present. -
Result: REQUEST_CHANGES
-
Reason: coverage-evidence result was
failure, so required test/docstring evidence was not proven for current head337f42601c214036fa4d4a55b7a29b710f4a4d2b. -
Head SHA:
337f42601c214036fa4d4a55b7a29b710f4a4d2b -
Workflow run: 30980475135
-
Workflow attempt: 1
Coverage evidence
Coverage Decision
- Result: FAIL
- Test evidence: not proven passing
- Docstring evidence: not proven passing when configured
- Failure count: 1
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Workflow (6 files)"]
S1 --> I1["GitHub Actions review job"]
I1 --> R1["Review risk: Workflow (6 files)"]
R1 --> V1["actionlint plus required checks"]
Evidence --> S2["Changed file (3 files)"]
S2 --> I2["repository behavior"]
I2 --> R2["Review risk: Changed file (3 files)"]
R2 --> V2["required checks"]
Evidence --> S3["Docs (5 files)"]
S3 --> I3["operator or user guidance"]
I3 --> R3["Review risk: Docs (5 files)"]
R3 --> V3["docs review"]
Evidence --> S4["CI script (7 files)"]
S4 --> I4["review and security gate shell path"]
I4 --> R4["Review risk: CI script (7 files)"]
R4 --> V4["bash -n plus Strix self-test"]
Evidence --> S5["Test (18 files)"]
S5 --> I5["regression suite"]
I5 --> R5["Review risk: Test (18 files)"]
R5 --> V5["targeted test run"]
|
Exact-head stack correction: the live branch head is |
|
@opencode-agent @cwl-noema-review @coderabbitai review Review the current exact head only after all current-head checks finish. Verify the review control plane end to end: fail-closed current-head evidence, reviewer identity and approval freshness, stale/queued/predecessor evidence rejection, NVIDIA NIM/OpenCode execution without |
|
I will verify the control plane, evidence freshness, reviewer identity, stale-evidence rejection, NVIDIA NIM/OpenCode token handling, output limits and redaction, coverage and docstrings, adversarial tests, APA 7 documentation, and protected-branch merge controls. I will report only current actionable findings.
|
|
@opencode-agent @cwl-noema-review Exact head remains |
|
Reconcile this control-plane branch with current protected |
|
Superseded by focused current- |
2 participants
Purpose
Restore the central review/security control plane, add a trusted pull-request comment invocation surface for the existing review agents, and consolidate the validated runtime-coverage and sandbox-evidence boundaries developed in focused replacement PRs.
Comment invocation
@cwl-noema-reviewdispatches the centralnoema-reviewevent with the live PR number and immutable current head SHA.@opencode-agentdispatches the central merge scheduler in review-only mode:enable_auto_merge=false,update_branches=false, andmerge_mode=disabled.issue_commentevents to sibling repositories.Central control-plane baseline
The branch restores the validated organization baseline:
JavaScript runtime coverage boundary
The Istanbul post-processor distinguishes product runtime modules from bounded non-product tooling without repository-specific exceptions:
scripts/check-*andscripts/verify-*commands outsidesrcremain governed by command-level CI contracts;*.config.ts, and every module undersrc/scriptsremain blocking runtime scope;coverage-final.jsonfails closed;Sandboxed evidence redaction
One production boundary redacts completed and timed-out process output, structured and echoed commands, sensitive option values, assignments, evidence notes, service tails, nested JSON values, and credential-shaped JSON keys before publication.
The implementation additionally:
shell=Falsewithout mutating the command being verified;A same-head regression discovered that the deep-recursion fallback leaked
api_key=secret. The failing contract was preserved first, then production parsing was repaired.Security and operational boundaries
OWNER,MEMBER, andCOLLABORATORcomments on open pull requests are accepted.github-actions[bot]with bot type.Scheduled-agent credential boundary
The write-capable hourly OpenCode repair path must use
NVIDIA_NIM_API_KEY, must not useCOPILOT_GITHUB_TOKEN, and must preserve the existing independent read-only OpenCode and Noema reviewer credential names and scopes. Draft PR #769 remains the NVIDIA-NIM-only scheduled autofix integration stacked on this branch until the central control-plane base stabilizes.Verification
The reconciled exact head must prove:
Older-head verdicts are not completion evidence.
Standards and doctoring
CHANGELOG.mdand APA 7 doctoring records cover review-agent invocation, hourly repair, central security baselines, JavaScript runtime coverage classification, and sandbox command/output redaction. Sources include current primary GitHub and Vitest documentation, NIST SSDF publications, OWASP logging guidance, and CWE material. No formal conformance is claimed.Stack and merge gate
uv.lockmaterialization PR fix(coverage): materialize trusted uv lock dependencies #743 must merge first.mainand rerun every gate on its new exact head and base.Merge remains prohibited until the reconciled exact head passes every required check and security gate, receives current-head automated review and a qualifying independent
APPROVEDreview, has no actionable unresolved thread, and satisfies branch protection without bypass.