Skip to content

Add direct-bounty evidence checklist schema and validator (closes #686) - #688

Open
poaspergillus wants to merge 1 commit into
NSPG13:mainfrom
poaspergillus:evidence-checklist-686
Open

Add direct-bounty evidence checklist schema and validator (closes #686)#688
poaspergillus wants to merge 1 commit into
NSPG13:mainfrom
poaspergillus:evidence-checklist-686

Conversation

@poaspergillus

Copy link
Copy Markdown

Summary

This PR adds a concise, machine‑readable evidence checklist for direct coding bounties as described in #686.

Changes

  • JSON Schema (schemas/direct-evidence-checklist-v1.json)
    Draft‑2020‑12 schema that binds:

    • exact source commit
    • repository
    • subdirectory
    • pull request URL
    • check‑run URLs (HTTPS only, non‑empty)
    • artifact digest (sha256/sha512, valid hex)
    • rejects empty, mutable, or non‑HTTPS references
  • Validator script (scripts/validate-evidence-checklist.mjs)

    • Pure Node.js, zero dependencies
    • Self‑test with 6/6 passing (valid, missing field, invalid commit, invalid digest, non‑HTTPS URL, empty check runs)
    • Produces a compact output suitable for MCP/API inclusion (no secrets, no personal data)

Validation

node scripts/validate-evidence-checklist.mjs --self-test
# 6/6 tests passed

Closes #686

@NSPG13 NSPG13 left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The zero-dependency validator, compact output, and malformed-field self-tests are useful. The current shape does not yet satisfy #686's evidence boundary and has two schema/validator inconsistencies that can admit invalid evidence:

  1. It does not explicitly separate submission, verification, and payment evidence, and it has no canonical BountySettled proof field. A PR/check run must never be representable as payment evidence.
  2. artifact_digest.digest accepts either 64 or 128 hex characters regardless of algorithm; sha256 must require 64 and sha512 128.
  3. The JSON Schema rejects additional properties, but the handwritten validator silently accepts them, so the advertised contract and executable validation disagree.
  4. The PR says mutable artifact references are rejected, but no immutable artifact reference/URI is modeled or tested.

Please model three explicit evidence sections, require an immutable HTTPS artifact reference plus digest, bind digest length to algorithm, validate/reject unknown properties consistently, and add tests proving a check-run-only object cannot claim payment and that a mutable/non-HTTPS artifact reference fails. The schema should use the canonical agentbounties.app ID. This can become a useful independent PR after those fixes; no collaboration branch is needed unless you want maintainer help.

…ngth binding

- Separate submission, verification, and payment evidence sections
- Require immutable HTTPS artifact reference + digest
- Bind digest length to algorithm (sha256=64 hex, sha512=128 hex)
- Reject unknown properties in all objects
- Add test proving check-run-only object cannot claim payment
- Schema uses canonical agentbounties.org ID

Closes NSPG13#686
@NSPG13

NSPG13 commented Aug 4, 2026

Copy link
Copy Markdown
Owner

This PR remains in manual-security review with requested changes. What passed: it targets issue #686 with a concrete validator. What blocks main: the checklist must be deterministic, versioned, and separate from payment authority, and it overlaps #739, #745, and #759. Action: resolve existing review threads, consolidate duplicate scope, run cargo run -p cli -- docs-contract-check and focused validator tests, and add invalid/valid evidence fixtures. Done when one canonical checklist is selected. Thanks for helping the community grow, and sorry for the review friction.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[DIRECT] Add a concise direct-bounty evidence checklist

2 participants