Summary
confidence_baseline today is a float the record's author assigns.
Nothing external verifies it, it's self-reported, the same shape whether
the underlying evidence is a formally disclosed CVE or a speculative
pattern match. This has been raised independently by two people, from
two unrelated angles, without either prompting the other, worth taking
seriously specifically because of that convergence.
The two independent hits
From a supply-chain attestation engineering angle (astrogilda,
issue #94): AVE's confidence_baseline is "a float the record's author
assigns," with no structural counterpart in the AEE predicate he's
proposing to crosswalk. In his own words from the issue body: "The
predicate has no confidence field and won't gain one. It expresses trust
structurally instead, through who stands behind the observation and
whether the observer saw the event or inferred it."
From a compliance/governance angle (Reddit, r/ai_governance,
VentureandCode_LEGAL): "clear guidelines on how tools should handle edge
cases or confidence scores would also be helpful to prevent compliance
logs from getting flooded with false positives." A compliance team
building workflows on top of AVE-classified findings needs to trust that
number enough to act on it, or filter on it, and a self-reported float
with no documented handling for edge cases is a real operational risk
once findings feed into an audit trail, not just a research artifact.
Neither of these is the same critique restated, one is about structural
verifiability, the other about operational trust once findings enter a
compliance workflow, but they land on the same underlying gap from
opposite directions.
What this issue is and isn't
This is a design question, not a proposed fix. Two shapes worth
considering, not yet chosen between:
- Document the handling, don't change the field. Write down
explicitly how confidence_baseline should be interpreted, what a
consuming tool should do at each band, how disagreement between
scanner-reported confidence and this baseline should be resolved.
Lower cost, doesn't touch the schema.
- Add structural backing. Something like an optional
verification_basis field (self-reported vs. externally attested vs.
independently cross-validated), giving confidence something more than
an assigned number to point at when it exists. Real schema change,
belongs in a deliberate version bump, not bundled in reactively.
Status
Not urgent, not blocking anything currently shipping. Opened now so the
convergence is visible and tracked, rather than sitting as two
acknowledgments in two separate comment threads where the pattern isn't
visible to anyone deciding what to prioritize next.
Summary
confidence_baselinetoday is a float the record's author assigns.Nothing external verifies it, it's self-reported, the same shape whether
the underlying evidence is a formally disclosed CVE or a speculative
pattern match. This has been raised independently by two people, from
two unrelated angles, without either prompting the other, worth taking
seriously specifically because of that convergence.
The two independent hits
From a supply-chain attestation engineering angle (astrogilda,
issue #94): AVE's
confidence_baselineis "a float the record's authorassigns," with no structural counterpart in the AEE predicate he's
proposing to crosswalk. In his own words from the issue body: "The
predicate has no confidence field and won't gain one. It expresses trust
structurally instead, through who stands behind the observation and
whether the observer saw the event or inferred it."
From a compliance/governance angle (Reddit, r/ai_governance,
VentureandCode_LEGAL): "clear guidelines on how tools should handle edge
cases or confidence scores would also be helpful to prevent compliance
logs from getting flooded with false positives." A compliance team
building workflows on top of AVE-classified findings needs to trust that
number enough to act on it, or filter on it, and a self-reported float
with no documented handling for edge cases is a real operational risk
once findings feed into an audit trail, not just a research artifact.
Neither of these is the same critique restated, one is about structural
verifiability, the other about operational trust once findings enter a
compliance workflow, but they land on the same underlying gap from
opposite directions.
What this issue is and isn't
This is a design question, not a proposed fix. Two shapes worth
considering, not yet chosen between:
explicitly how confidence_baseline should be interpreted, what a
consuming tool should do at each band, how disagreement between
scanner-reported confidence and this baseline should be resolved.
Lower cost, doesn't touch the schema.
verification_basisfield (self-reported vs. externally attested vs.independently cross-validated), giving confidence something more than
an assigned number to point at when it exists. Real schema change,
belongs in a deliberate version bump, not bundled in reactively.
Status
Not urgent, not blocking anything currently shipping. Opened now so the
convergence is visible and tracked, rather than sitting as two
acknowledgments in two separate comment threads where the pattern isn't
visible to anyone deciding what to prioritize next.