fix(security): enforce discovery and analysis completeness - #410
fix(security): enforce discovery and analysis completeness#410Spectorian wants to merge 11 commits into
Conversation
Signed-off-by: Nir Paz <npaz@nvidia.com>
rng1995
left a comment
There was a problem hiding this comment.
Requesting changes because the requested semantic pass can still be skipped entirely while this PR reports it as used and complete, producing an install-safe verdict. The current-head end-to-end repro executes zero semantic calls but returns llm_used=True, scan_mode=static+llm, recommendation=SAFE, and safe_to_install=True.
| if not enabled: | ||
| return False, False | ||
| call_log = result.get("llm_call_log") | ||
| if not isinstance(call_log, list) or not call_log: |
There was a problem hiding this comment.
[P1] Do not count an empty call log as a completed LLM pass. If the graph is imported before provider credentials are available, credential-gated semantic nodes are omitted. Binding a supported provider later through use_provider makes preflight succeed, but no semantic nodes or calls run and llm_call_log remains empty. This branch then returns (used=True, complete=True), allowing a SAFE/safe_to_install=True result even though meta_analysis_applied=False. Treat an empty log as not used/incomplete, or retain the expected analyzers and account their explicit statuses before allowing an install approval.
Signed-off-by: Nir Paz <npaz@nvidia.com>
Signed-off-by: Nir Paz <npaz@nvidia.com>
Signed-off-by: Nir Paz <npaz@nvidia.com>
Signed-off-by: Nir Paz <npaz@nvidia.com>
Signed-off-by: Nir Paz <npaz@nvidia.com>
|
Powered by Codex: correction after a five-lens GPT-5.6-sol council on current head
This supersedes the prior mergeable assessment. New, non-duplicate findings:
flowchart LR
A[".hidden-skill/SKILL.md"] --> B["dot-child discovery"]
B --> C["recursive re-root"]
C --> D["path becomes SKILL.md"]
D --> E["llm_file_cache"]
E --> F["external semantic provider"]
New anchored review: #410 (review) Recommended action: do not merge until hidden/local-only provenance survives recursive re-rooting and semantic incompleteness is source-aware and canonical. |
|
Fixes pushed. please review and confirm. |
mohgupta-ship-it
left a comment
There was a problem hiding this comment.
Powered by Codex: five-lens GPT-5.6-sol delta council — REQUEST CHANGES on 1b252b5. The prior mergeable assessment is superseded. The new dot-child path crosses the documented hidden-content/LLM trust boundary, and semantic completeness is inconsistent across transitive reporting and the CLI gate.
| except OSError as exc: | ||
| raise _read_error("multi_skill_directory_entry") from exc | ||
| if entry.name in _SKIP_DIRS or entry.name.startswith("."): | ||
| if entry.name in _SKIP_DIRS: |
There was a problem hiding this comment.
Powered by Codex: [P0] Including a dot-prefixed child and then recursively re-rooting it erases the hidden path segment used by build_context to enforce local-only handling. A harmless marker was excluded from llm_file_cache as .review-helper/SKILL.md in the parent, but appeared in llm_file_cache as SKILL.md after child re-rooting. This contradicts the documented invariant that hidden content is never sent to an external LLM. Carry immutable root-relative hidden/local-only provenance into the child scan and add a mocked-provider test proving the marker never reaches an LLM batch.
|
|
||
| for analyzer_id in required_analyzer_ids: | ||
| statuses = statuses_by_analyzer.get(analyzer_id) | ||
| if statuses is None or len(statuses) != 1: |
There was a problem hiding this comment.
Powered by Codex: [P1] This global cardinality check rejects valid transitive telemetry. One successful root scope returns (used=True, complete=True); adding one equally successful child scope returns (True, False) because both retain the same analyzer IDs. Validate one terminal status per (source_identity, analyzer_id) (and source-bind call records), or validate each child before flattening. Keep true duplicate-within-scope rejection.
| result=state, | ||
| discovered_modules=ANALYZER_MODULES, | ||
| ) | ||
| semantic_runtime_incomplete = bool(llm_requested and use_llm and not semantic_runtime_complete) |
There was a problem hiding this comment.
Powered by Codex: [P1] Semantic-runtime failure is only reflected in degraded metadata/recommendation, not canonical analysis_completeness. With requested analysis and missing telemetry, the report is CAUTION and llm_degraded=True but JSON still says is_complete=True/status=complete; consequently --fail-on-incomplete can exit successfully. Project this failure into the canonical ledger/completeness result before rendering and add CLI exit-code coverage.
Summary
Validation
git diff --check