feat: evidence-grade NIM discovery + all-modality cost-quality benchmark - #90
feat: evidence-grade NIM discovery + all-modality cost-quality benchmark#90seonghobae wants to merge 159 commits into
Conversation
…ark (#86) Optional stdlib-only benchmark harness (contextual_orchestrator/nim_benchmark.py): - Dynamic catalog discovery from the OpenAI-compatible GET /v1/models — no hard-coded inventory; deduplicated, sorted (response-order-drift immune), with machine-readable duplicate/invalid hygiene lists. - Capability probes for every contract NIM can host: chat completions, text completions, Responses API, embeddings, image understanding, video understanding, omni-style audio understanding, audio transcription, and audio speech — omni_capable derived, skipped probes always carry a machine-readable reason, bounded concurrency under one shared hard request budget. - Fair policy comparison on a locked task split: per-worker direct baselines (source of best-single-worker-in-hindsight), route_once, conduct capped at five steps, cheapest-eligible-worker; identical scorers, caps, timeouts, and token budgets across systems. - Honest cost accounting: actual cost 0 while the hosted catalog is free; hypothetical paid cost only from an explicit versioned pricing scenario, "unknown" otherwise; the two never mix. - Paired-bootstrap CIs, quality-latency and quality-hypothetical-cost Pareto frontiers, full provenance (git SHA, run id, catalog/manifest/ pricing hashes, parameters), schema-validated JSON/CSV/Markdown artifacts with a secret-leak refusal guard. - Fail closed: missing KV credential (NVIDIA_NIM_API_KEY, bootstrap env->KV only, never argv), incomplete discovery, exceeded budget, missing provenance, schema violations. - Deterministic --dry-run drives the whole pipeline against an in-process synthetic provider covering every modality class — zero network, byte-identical artifacts. - Tests: 100% statement+branch coverage of the new module, adversarial cases (malformed catalogs, duplicate ids, non-finite tokens/costs, rate limits, timeouts, order drift, secret redaction); new fuzz seam (Hypothesis + Atheris) for the catalog parser. - CI: manual + conservative monthly scheduled workflow, single-flight concurrency, hard budgets, pinned actions, 90-day artifact retention. - Docs: docs/nim_benchmark.md, architecture/tracks pointers, HELM (arXiv:2211.09110) added to docs/papers. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
📝 WalkthroughWalkthroughNVIDIA NIM 비용·품질 벤치마크 하네스를 추가했습니다. 카탈로그 탐색, modality 검증, 정책 평가, 비용·증거 검증, 결정적 dry-run, live CI 실행, artifact 생성 및 품질 테스트를 포함합니다. ChangesNIM 벤치마크 기능과 검증
Estimated code review effort: 5 (Critical) | ~120 minutes Possibly related PRs
Sequence Diagram(s)sequenceDiagram
participant Operator
participant GitHubActions
participant NIMBenchmark
participant NIMProvider
participant ArtifactStore
Operator->>GitHubActions: dry-run 또는 live 실행 요청
GitHubActions->>NIMBenchmark: 고정된 예산·manifest·provenance 전달
NIMBenchmark->>NIMProvider: 카탈로그 조회 및 capability probe
NIMProvider-->>NIMBenchmark: 모델·probe 결과 반환
NIMBenchmark->>NIMProvider: 평가 task 요청 전송
NIMProvider-->>NIMBenchmark: 응답·usage 반환
NIMBenchmark->>ArtifactStore: JSON·CSV·Markdown artifact 업로드
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 2📝 Generate docstrings 💡
⚔️ Resolve merge conflicts 💡
🧪 Generate unit tests (beta)
Comment |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
|
Exact-head maintainer audit of
Do not mark this PR ready or resolve these findings from stale checks. Re-run all required workflows and independent review on the exact repaired head. |
seonghobae
left a comment
seonghobae
left a comment
There was a problem hiding this comment.
Blocking security finding
CRITICAL — benchmark HTTPS transport reintroduces DNS-rebinding SSRF
require_public_https_endpoint() validates one DNS answer, but build_default_transport() then calls urllib.request.urlopen(), which resolves the hostname again. The validated_hosts cache widens the gap by skipping validation on later calls. A provider hostname can therefore pass public-address validation and later connect to loopback, private, RFC 6598, link-local, or otherwise non-global infrastructure. Redirects are also delegated to urllib's default handler, so credentials may be forwarded to an unvalidated destination.
This is the same TOCTOU class repaired by PR #76. Keep this PR Draft until #76 is in main, then reuse or generalize its DNS-pinned transport so the socket dials only validation-time public IPs while preserving the original hostname for HTTP authority, TLS SNI, and certificate verification. Reject redirects, bypass environment proxies, require is_global, clear stale pins before each validation, and deterministically close failed sockets/responses.
Required regression evidence:
- no transport-time DNS re-resolution;
- public IPv4/IPv6 pinning and hostname/SNI preservation;
- RFC 6598/private/loopback/link-local/multicast/reserved/unspecified rejection;
- redirect rejection with no authorization propagation;
- approved-IP fallback and all-address failure;
- exact-head statement/branch coverage and full repository checks.
Do not merge based on the current local 100% claim; it does not cover this security invariant.
seonghobae
left a comment
seonghobae
left a comment
There was a problem hiding this comment.
Exact-head review — still blocked
The latest head addresses the three audit themes, but the implementation is not yet mergeable.
1. HIGH — production security behavior is coupled to a test monkeypatch
nim_benchmark_hardening._build_secure_transport() deliberately falls back to urllib.request.urlopen() whenever that global has been replaced. This keeps the old offline tests green by changing production behavior instead of replacing the obsolete tests. It reintroduces the hostname-resolving, proxy-aware, redirect-capable path that this repair is supposed to eliminate, and requires a nosemgrep waiver on the exact sink.
Remove the compatibility branch entirely. Rewrite the transport tests to inject the pinned connection/resolver seam and prove the real direct transport. Do not let test instrumentation select a less-secure production path.
2. HIGH — the new 531-line hardening module has no direct regression suite
The current tests/test_nim_benchmark.py still asserts the old urlopen behavior and contains no evidence for EqualBudgetModelClient, expiry enforcement, actual-cost evidence validation, configured/observed budget fields, or the installed wrappers. A 100% repository claim cannot be accepted while the newly shipped module and its branches are not directly exercised.
Add behavior tests that fail without each contract, then prove exact-head statement and branch coverage at 100% for both nim_benchmark.py and nim_benchmark_hardening.py, plus 100% public docstrings.
3. HIGH — cited cost source does not support the recorded claim
ACTUAL_COST_EVIDENCE names https://docs.nvidia.com/nim/large-language-models/latest/faq.html, but the current NVIDIA FAQ location is different and the available FAQ content does not establish that the API Catalog hosted endpoint used by this run is free to the caller. Current build.nvidia.com model pages do label prototype endpoints as free, while NIM offering/licensing documentation distinguishes free exploratory offerings from NVIDIA AI Enterprise production support. Record the exact reviewed page/version or immutable evidence artifact that supports the hosted endpoint claim; do not cite a generic or moved FAQ as proof.
4. MEDIUM — optional adapter boundary is lost
contextual_orchestrator.__init__ now imports the entire benchmark and monkeypatch installer for every package import. Keep the NIM evaluator optional: integrate the fix directly into its module, or install it only from the benchmark entrypoint. Runtime gateway users should not import benchmark-only code or mutate benchmark globals as a side effect of import contextual_orchestrator.
5. MEDIUM — temporary privileged workflow remains in the PR tree
.github/workflows/temporary-nim-transport-fix.yml has top-level contents: write, persists checkout credentials, and triggers on every branch push. It must be removed from the final tree. Any one-shot repair must use read-only defaults, isolate credentials to the publication step, validate exact changed paths, and disappear before review.
Keep the PR Draft. Replace the compatibility installer with direct, test-first implementation; remove the temporary workflow; update the PR body to the actual head; then rerun all exact-head checks and independent review.
|
Exact-head maintainer follow-up for the new hardening commits:
Do not resolve the security thread or mark the PR ready until these are implemented and verified on the exact repaired head. |
|
Exact-head update after CI-driven repairs:
The first exact-head verification attempt correctly failed: the existing workflow contract required the original one-line |
|
Exact-head verification receipt for
These results verify the exact feature head against temporary verification base |
|
Correction to the earlier verification receipt: runs 31040728716, 31040728278, 31040728943, 31040728271, and 31040728426 checked out synthetic merge commit 7e89420, not branch head 88b0e96. They are integration-smoke evidence only and do not satisfy exact-head gates. Current head 692e49a adds a regression contract and explicitly selects the pull-request head SHA in every repository-local Tests, Fuzz, and Security checkout. Fresh job logs must show that exact SHA before any result is described as exact-head evidence. |
|
Exact-head repository-native verification is complete for current feature head
This records repository-local current-head evidence only. It does not satisfy the post-#96 integrated-base rerun, current-head automated review, independent non-author approval, branch protection, or release gates. Verification PR #101 will be closed unmerged as intended. |
|
Exact-current-head review request for @opencode-agent @cwl-noema-review @coderabbitai review |
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
HEAD_SHA="f0429534618c283a40ee841d61a13024c9455b9f"
BASE_SHA="8b479f5667e92b39a6c611cb58cfa6f39a9cfa0f"
printf '%s\n' '== Checked-out commit =='
git rev-parse HEAD
git show -s --format='HEAD=%H%nparents=%P%nsubject=%s' HEAD
printf '%s\n' '== Requested commits =='
git cat-file -t "$HEAD_SHA"
git cat-file -t "$BASE_SHA"
git merge-base --is-ancestor "$BASE_SHA" "$HEAD_SHA" && echo "base_is_ancestor=true" || echo "base_is_ancestor=false"
git diff --check "$BASE_SHA...$HEAD_SHA"
git diff --name-status "$BASE_SHA...$HEAD_SHA"
printf '%s\n' '== Relevant source and test outlines =='
ast-grep outline contextual_orchestrator/nim_benchmark.py --items all --view expanded
for candidate in \
tests/test_nim_benchmark.py \
tests/test_nim_benchmark_review_regressions.py \
tests/test_nim_benchmark_release_acceptance.py \
tests/test_nim_benchmark_workflow_contract.py \
tests/test_nim_benchmark_workflow_secret_boundary.py \
fuzz/targets.py \
fuzz/fuzz_nim_catalog.py; do
if [ -f "$candidate" ]; then
echo "--- $candidate"
ast-grep outline "$candidate" --items all --view expanded
fi
done
printf '%s\n' '== Security, planning, provenance, artifact, and import-boundary references =='
rg -n -C 3 --type py \
'class Pinned|Pinned|NoRedirect|ProxyHandler|urlopen|create_connection|getaddrinfo|CAPABILITY_PROBE_ORDER|plan_complete_request_budget|probe_discovered_models|BenchmarkBudgetError|remaining_requests|conduct|route_once|actual_cost_usd|hypothetical_cost_source|pricing|evidence|pareto|excluded_zero_success|write_benchmark_artifacts|os\.replace|atomic|RecursionError|_REPORT_REQUIRED_PATHS|timeout_seconds' \
contextual_orchestrator/nim_benchmark.py tests fuzz
printf '%s\n' '== Import and workflow boundary references =='
rg -n -C 3 --type py 'nim_benchmark|nim_benchmark_hardening' contextual_orchestrator/__init__.py contextual_orchestrator/__main__.py tests 2>/dev/null || true
for workflow in .github/workflows/tests.yml .github/workflows/fuzz.yml .github/workflows/security.yml .github/workflows/nim-benchmark.yml; do
if [ -f "$workflow" ]; then
echo "--- $workflow"
sed -n '1,280p' "$workflow"
fi
done
printf '%s\n' '== Temporary artifacts and prohibited compatibility paths =='
for candidate in \
.review-evidence/nim-direct-integration \
.review-evidence/nim-source-repair.yml \
contextual_orchestrator/nim_benchmark_hardening.py \
tests/test_nim_benchmark_hardening.py; do
if [ -e "$candidate" ]; then
echo "PRESENT $candidate"
else
echo "ABSENT $candidate"
fi
done
printf '%s\n' '== Exact-head GitHub check runs =='
gh api "repos/ContextualWisdomLab/contextual-orchestrator/commits/$HEAD_SHA/check-runs?per_page=100" \
--jq '.check_runs[] | [.name, .status, .conclusion, .details_url] | `@tsv`' || true
printf '%s\n' '== Pull request metadata and review state =='
gh pr view 90 --repo ContextualWisdomLab/contextual-orchestrator \
--json number,state,isDraft,headRefOid,baseRefOid,statusCheckRollup,reviews \
--jq '{number, state, isDraft, headRefOid, baseRefOid, checks: [.statusCheckRollup[] | {name, status, conclusion}], reviews: [.reviews[] | {author: .author.login, state, commit: .commit.oid}]}' || trueLength of output: 178131 🏁 Script executed: #!/bin/bash
set -euo pipefail
HEAD_SHA="f0429534618c283a40ee841d61a13024c9455b9f"
BASE_SHA="8b479f5667e92b39a6c611cb58cfa6f39a9cfa0f"
printf '%s\n' '== Exact revision and changed paths =='
printf 'checked_out='; git rev-parse HEAD
git show -s --format='head=%H%nparents=%P%nsubject=%s' "$HEAD_SHA"
git merge-base "$BASE_SHA" "$HEAD_SHA"
git diff --name-status "$BASE_SHA...$HEAD_SHA"
git diff --stat "$BASE_SHA...$HEAD_SHA"
printf '%s\n' '== NIM benchmark function locations =='
rg -n --type py \
'^(def (build_default_transport|discover_model_catalog|probe_discovered_models|plan_complete_request_budget|planned_complete_run_requests|evaluate_policies|build_pareto_frontiers|render_markdown_summary|validate_report_schema|write_benchmark_artifacts|assemble_benchmark_report|run_benchmark)|class (_BudgetedModelClient|EqualBudgetModelClient|RequestBudget|_PinnedHTTPSConnection))\b|^_REPORT_REQUIRED_PATHS\b|^CAPABILITY_PROBE_ORDER\b|^MAX_WORKFLOW_DEPTH\b' \
contextual_orchestrator/nim_benchmark.py
printf '%s\n' '== Focused implementation sections =='
python - <<'PY'
from pathlib import Path
import re
path = Path("contextual_orchestrator/nim_benchmark.py")
lines = path.read_text(encoding="utf-8").splitlines()
names = (
"build_default_transport", "discover_model_catalog", "probe_discovered_models",
"plan_complete_request_budget", "planned_complete_run_requests", "evaluate_policies",
"build_pareto_frontiers", "validate_report_schema", "render_markdown_summary",
"write_benchmark_artifacts", "assemble_benchmark_report", "run_benchmark",
)
starts = []
for index, line in enumerate(lines, 1):
if re.match(rf"^def ({'|'.join(names)})\(", line):
starts.append(index)
starts.append(len(lines) + 1)
for start, end in zip(starts, starts[1:]):
print(f"\n--- {path}:{start}-{end - 1} ---")
print("\n".join(f"{n:5} {lines[n-1]}" for n in range(start, end)))
PY
printf '%s\n' '== Markdown direct report paths versus schema paths =='
python - <<'PY'
import ast
from pathlib import Path
source = Path("contextual_orchestrator/nim_benchmark.py").read_text(encoding="utf-8")
tree = ast.parse(source)
required = set()
render = None
for node in tree.body:
if isinstance(node, ast.Assign) and any(
isinstance(target, ast.Name) and target.id == "_REPORT_REQUIRED_PATHS"
for target in node.targets
):
required = set(ast.literal_eval(node.value))
if isinstance(node, ast.FunctionDef) and node.name == "render_markdown_summary":
render = node
def path_of(node):
parts = []
while isinstance(node, ast.Subscript):
key = node.slice
if isinstance(key, ast.Constant) and isinstance(key.value, str):
parts.append(key.value)
else:
return None
node = node.value
if isinstance(node, ast.Name) and node.id == "report":
return ".".join(reversed(parts))
return None
paths = sorted({path for node in ast.walk(render) if (path := path_of(node))})
print("required paths:")
print("\n".join(sorted(required)))
print("\ndirect report subscript paths:")
print("\n".join(paths))
print("\nmissing from required paths:")
print("\n".join(path for path in paths if path not in required) or "<none>")
PY
printf '%s\n' '== Provider transport ownership and benchmark imports =='
fd -a 'provider_transport\.py|nim_benchmark\.py|nim_csv_evidence\.py' contextual_orchestrator -x sh -c 'echo "--- $1"; sed -n "1,320p" "$1"' sh {}
rg -n -C 2 --type py \
'urllib\.request\.urlopen|build_opener|ProxyHandler|HTTPRedirectHandler|socket\.getaddrinfo|create_connection|server_hostname|NVIDIA_NIM_API_KEY|nim_benchmark' \
contextual_orchestrator/nim_benchmark.py contextual_orchestrator/provider_transport.py contextual_orchestrator/__init__.py contextual_orchestrator/__main__.py
printf '%s\n' '== Exact-head statuses and required branch checks =='
gh api "repos/ContextualWisdomLab/contextual-orchestrator/commits/$HEAD_SHA/status" \
--jq '{state, statuses: [.statuses[] | {context, state, target_url}]}' || true
gh api "repos/ContextualWisdomLab/contextual-orchestrator/branches/main/protection/required_status_checks" \
--jq '{strict, contexts, checks}' || true
gh api "repos/ContextualWisdomLab/contextual-orchestrator/rules/branches/main" \
--jq '.' || trueLength of output: 166515
I reviewed exact head I found one blocking artifact-integrity defect.
The requested repository-local exact-head checks are successful. These include full tests, NIM 100% coverage and docstring gates, wheel install/import, Hypothesis, Atheris, CodeQL, and Python supply-chain checks. I could not verify all organization-central required gates. The organization rules require Do not approve this Draft PR. Resolve the atomic-publication defect and provide successful exact-head results for the organization-central required workflows. You are interacting with an AI system. |
|
@opencode-agent @cwl-noema-review @coderabbitai review Please independently review exact current head |
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
head_sha="f0429534618c283a40ee841d61a13024c9455b9f"
base_sha="8b479f5667e92b39a6c611cb58cfa6f39a9cfa0f"
repo="ContextualWisdomLab/contextual-orchestrator"
echo "== Revision =="
printf "checked_out="; git rev-parse HEAD
git show -s --format='head=%H%nparents=%P%nsubject=%s' "$head_sha"
git merge-base --is-ancestor "$base_sha" "$head_sha" && echo "base_is_ancestor=true"
git diff --check "$base_sha...$head_sha"
echo "== Artifact implementation map =="
ast-grep outline contextual_orchestrator/nim_benchmark.py --items all --view expanded
rg -n -C 8 --type py \
'def write_benchmark_artifacts\(|def validate_report_schema\(|def render_markdown_summary\(|os\.replace|tempfile|mkdtemp|TemporaryDirectory|benchmark_report\.json|benchmark_cells\.csv|benchmark_summary\.md|enrich_benchmark_cell_csv' \
contextual_orchestrator tests
echo "== Artifact implementation =="
python - <<'PY'
from pathlib import Path
import ast
path = Path("contextual_orchestrator/nim_benchmark.py")
source = path.read_text(encoding="utf-8")
tree = ast.parse(source)
lines = source.splitlines()
for node in tree.body:
if isinstance(node, ast.FunctionDef) and node.name == "write_benchmark_artifacts":
for index in range(node.lineno, node.end_lineno + 1):
print(f"{index:5} {lines[index - 1]}")
PY
echo "== Relevant tests =="
fd -a 'test_nim*.py' tests -x sh -c '
echo "--- $1"
rg -n -C 5 "write_benchmark_artifacts|artifact|atomic|os\.replace|TemporaryDirectory|mkdtemp" "$1" || true
' sh {}
echo "== Exact-head check runs =="
gh api "repos/$repo/commits/$head_sha/check-runs?per_page=100" \
--jq '.check_runs[] | [.name, .status, (.conclusion // "null"), .head_sha, .details_url] | `@tsv`' || true
echo "== PR state and check rollup =="
gh pr view 90 --repo "$repo" \
--json isDraft,headRefOid,baseRefOid,statusCheckRollup \
--jq '{isDraft, headRefOid, baseRefOid, checks: [.statusCheckRollup[] | {name, status, conclusion, detailsUrl}]}' || true
echo "== Required checks and rules =="
gh api "repos/$repo/branches/main/protection/required_status_checks" \
--jq '{strict,contexts,checks}' || true
gh api "repos/$repo/rules/branches/main" --jq '.' || trueLength of output: 73982
I found one blocking artifact-integrity defect.
Stage, validate, and enrich the complete artifact set outside the final output directory. Publish the complete set only after all operations succeed. On failure, do not leave newly created final artifacts. The exact-head repository checks are successful. This includes Tests, Fuzz, Security, CodeQL, the NIM 100% coverage/docstring gate, and package smoke checks. I cannot verify all organization-required gates. The organization rules require I do not approve this Draft PR. You are interacting with an AI system. |
|
Implement the current-head CodeRabbit blocker on branch Add RED regression coverage proving that a failure during CSV enrichment/rendering or Markdown generation leaves no newly published final artifact set, does not corrupt a previously complete artifact set, and removes all staging/backup residue. Then implement a bounded cross-platform transaction for After GREEN, run the focused artifact tests, full benchmark coverage/docstring/package gates, Tests/Fuzz/Security, and request the organization-central security/review workflows for the resulting exact head. Do not resolve the blocker or mark Ready until the exact resulting SHA has successful required checks and a fresh independent review. |
seonghobae
left a comment
seonghobae
left a comment
There was a problem hiding this comment.
Exact-head blocker recorded as an unresolved inline thread because the author account cannot submit REQUEST_CHANGES on its own PR. Keep Draft until addressed and reverified.
| """Validate cost evidence and schema, then write JSON, CSV, and Markdown.""" | ||
| _validate_actual_cost_evidence(report) | ||
| validate_report_schema(report) | ||
| os.makedirs(output_dir, exist_ok=True) |
There was a problem hiding this comment.
BLOCKER — JSON/CSV/Markdown are published as a partial set. Creating the final directory here and then writing each artifact directly means a later CSV enrichment/rendering, Markdown, or filesystem failure can leave a newly visible partial set or partially replace a prior complete set. Repair test-first: stage, validate, render, secret-scan, and enrich the complete set outside the final path; publish only after all steps succeed; preserve/restore an existing complete set on ordinary publication failure; remove staging/backup residue; and document the exact portable rollback/crash-window contract. Add regressions for fresh-target failure, prior-set preservation, mid-publication failure, cleanup, and returned paths. Do not resolve until the repaired exact head has 100% statement/branch/docstring evidence and all required checks.
There was a problem hiding this comment.
Addressed test-first on exact head 1cea0c019a0af0f523e5ffef10dfafcba4554baa through the supported CLI composition boundary. The branch now stages the complete JSON/CSV/Markdown set in a hidden sibling directory, enriches and validates the CSV there, validates exact non-empty regular-file membership, publishes by same-filesystem directory renames, restores a prior complete set after ordinary mid-publication failure, removes staging/backup residue, recovers a sole interrupted backup on the next invocation, rejects ambiguous backups, and rewrites success output to final public paths only. The portable two-rename crash window and recovery limits are documented. Focused regressions are in tests/test_nim_artifact_publication.py and tests/test_nim_artifact_publication_edges.py; the current exact head has successful repository Tests run 31073765240, Fuzz run 31073765223, and Security run 31073765231. The thread remains unresolved because trusted central 100% statement/branch/docstring/package evidence and every required current-head gate still depend on .github#759 integration and subsequent exact-head redispatch.
Exact-current-head revalidation recordInventory identity for this review cycle:
The artifact-publication blocker has been implemented test-first at the active CLI composition root without making package import eager. Exact-head repository evidence now observed:
This does not establish merge acceptance. No exact-head Security Scan or SAST Semgrep run is currently present in the observed workflow inventory, the inline blocker remains unresolved pending the complete gate, and a qualifying independent non-author approval is still required. Keep this PR Draft and do not reuse predecessor-head or synthetic-merge evidence. |
2 participants
Purpose
Implement issue #86 as a provider-neutral, evidence-grade NVIDIA NIM discovery and benchmark harness while preserving standalone operation and the modular CWL MSA boundary.
Exact integration identity
1cea0c019a0af0f523e5ffef10dfafcba4554baafix/atheris-interpreter-lock8b479f5667e92b39a6c611cb58cfa6f39a9cfa0f4926bdb12f22a6b45cf842ce35d48eb0d23c1769The branch descends from the exact current #96 head. Every check, review, or approval associated with a predecessor feature head is historical only and must not be reused.
Current-head verification state
Exact-head repository workflows completed successfully:
31073765240;31073765223;31073765231;Trusted organization-central 100% statement/branch/docstring/package evidence, Security Scan, SAST Semgrep, current-head OpenCode/Noema/Strix review, and qualifying independent non-author approval are not yet complete for this exact head. The central prerequisite remains
ContextualWisdomLab/.github#759; after it merges, #96 must regenerate authoritative exact-head coverage and review evidence, merge first, and this PR must then be retargeted to the integratedmainand fully revalidated. Queued, pending, cancelled, predecessor-head, stale-base, synthetic-merge, or absent evidence does not count as success.Implemented scope
GET /v1/modelscatalog without a hard-coded authoritative inventory;route_once, boundedconduct, and reviewed cheapest-worker policies under one common prompt-plus-completion token allowance and maximum-call envelope;unknown;CHANGELOG.mdentries.Security and import boundary
NVIDIA_NIM_API_KEY; only the bounded live job owns the GitHub Secret binding;Evidence contracts
Test-first regressions cover:
CatalogDiscoveryError;The dedicated quality gate requires 100% production statement and branch coverage for the benchmark and publication modules, 100% public docstrings, wheel build/install, and clean installed-package imports.
Unresolved exact-head gate
The transactional-publication blocker is implemented and documented, and repository Tests/Fuzz/Security pass on the exact head. Its inline thread intentionally remains unresolved until trusted central 100% statement/branch/docstring/package evidence and every required current-head gate complete, as required by the review instruction.
Stack order and merge acceptance
.github#759after exact-head independent approval;mainwithout changing the reviewed feature boundary;Do not mark Ready, merge, publish, or release until:
Refs #86.
Depends on #96 and ContextualWisdomLab/.github#759.