Skip to content

Repository files navigation

ResearchCrux

CI Spec Quality License: CC BY 4.0

Public evidence layer: protocol specs, proof gallery, regulatory mappings, API contracts.

Part of CueCrux. Where outsiders can verify claims without accessing the machinery.

ResearchCrux is the public evidence and protocol layer for the CueCrux platform. It contains protocol specifications, living benchmark evidence, proof galleries, and published API/MCP contracts, all generated from or grounded in the platform itself.

VaultCrux is infrastructure. CueCrux is the Reasoning OS. ResearchCrux is where outsiders can verify claims without accessing the machinery.


Version Matrix

Dimension Current Value
CROWN Protocol v0.1
SCITT Application Profile v0.2 (Pre-submission Review)
Receipt Schema 1.1 (llmModel + llmRequestId hash-bound)
Benchmark Phase 7.4 (12/12 x 5, 2026-03-24)
Audit Source audit-source/ -- runs 037b303a through fabf5dc8
Config Manifest 6.7 + schema 1.1

Source of truth: Benchmark methodology and canonical results are included in audit-source/ (mirrored from AuditCrux, MIT licensed). ResearchCrux publishes the evidence; the audit suite produces it.


Quick Navigation

Section Description Start Here
Protocol Specs CROWN receipt protocol specification, hash chain construction, IETF SCITT alignment CROWN v0.1
SCITT Compatibility CDDL schema, COSE encoding, registration policy, privacy considerations for SCITT integration Integration Guide
Living Evidence Benchmark ledger with historical runs, DQP findings, embedding comparison Ledger Index
MemoryCrux Benchmark Standard Normative spec: arms, scoring, controls, anti-gaming, reproducibility, governance Standard v1.0
MemoryCrux Cost Analysis Tool-mediated memory vs context-stuffing: subscription breakeven, TCO, quality-adjusted cost Cost Analysis
MemoryCrux Governance Benchmark council, decision authority, external rerun program, amendment process Charter
MemoryCrux Proof Packs Per-claim evidence bundles: safety, recall, cost — with run receipts and statistics Proof Packs
Proof Gallery CROWN receipt examples, redacted proof packs, verification walkthrough Gallery Index
Regulatory Mapping EU AI Act (Art. 13, 14) and DORA (Art. 8–11) mapped to CROWN capabilities Mapping
API & MCP Contracts Published API surface, MCP tool catalog, JSON schemas Contracts Index
Whitepapers Retrieval quality benchmark methodology and results Benchmark v1
Manifesto Product philosophy: "Receipts Over Vibes" Manifesto v2.3

Audiences

  • Compliance engineers evaluating EU AI Act (Article 13, 14) and DORA (Articles 8-11) obligations: regulatory-mapping.md
  • Standards participants evaluating CROWN as a SCITT application profile: SCITT integration guide
  • Technical evaluators who need measured performance and architecture evidence: benchmark ledger
  • Integrators building on the CueCrux API or MCP surface: contracts

Evidence Generation

This repository contains no runnable infrastructure. Evidence is generated from canonical audit runs using the generation scripts in this repo:

npm install
npm run generate          # regenerate all evidence from source
npm run ledger:generate   # benchmark ledger only
npm run proof:generate    # proof gallery only
npm run contracts:generate # API/MCP contract docs only

The benchmark suite methodology and canonical results are included in audit-source/ (MIT).


Canonical Results

Current Baseline: v4 Phase 7.4 (2026-03-24)

Suite Run ID Embedding Categories Result
v4 - Production (schema 1.1) 037b303a EmbedderCrux/nomic 12 categories 12/12
v4 - Production (schema 1.1) 80434381 EmbedderCrux/nomic 12 categories 12/12
v4 - Production (schema 1.1) 69341abe EmbedderCrux/nomic 12 categories 12/12
v4 - Production (schema 1.1) e0bfbd9b EmbedderCrux/nomic 12 categories 12/12
v4 - Production (schema 1.1) fabf5dc8 EmbedderCrux/nomic 12 categories 12/12

Corpus: 1074 unique docs, 462 queries, 12 categories. LLM: gpt-4o-mini (OpenAI). Schema: Receipt schema 1.1 (llmModel + llmRequestId hash-bound).

Key metrics across 5 canonical runs:

Category Metric Range Target
Cat 2 (Format-Aware Citation) avg_citation_recall 0.633-0.693 >=0.50
Cat 8 (Proposition Precision) precision_at_1 0.963 (5x) >=0.80
Cat 11 (Chunking Stress) broad_recall 0.927 (5x) >=0.70
Cat 12 (Hard-Negative Overlap) parent_child_recall 1.000 (5x) >=0.80

Phase 7.4 adds LLM metadata binding (schema 1.1) to CROWN receipts. Zero retrieval code changed. Cat 7, 8, 11, 12 are perfectly deterministic across all 5 runs. Cat 5 receipt chain 10/10 intact (schema 1.1 validated).

Legacy Suites (v1-v3)

Suite Run ID Embedding Categories Result
v1 - Baseline 110ada93 OpenAI 4 × 3 modes 12/12
v2 - Enterprise c85daff7 OpenAI 4 × 3 modes 12/12
v3 - Capability e782fbd0 OpenAI 6 × 3 modes 16/16
v1 - Baseline a86b1733 EmbedderCrux/nomic 4 × 3 modes 12/12
v2 - Enterprise 5b125495 EmbedderCrux/nomic 4 × 3 modes 12/12
v3 - Capability 8dd5efff EmbedderCrux/nomic 6 × 3 modes 16/16

See the benchmark ledger for per-run details, metric deltas, and downloadable evidence.


Citation

CueCrux. Retrieval Quality Benchmarking for Enterprise Knowledge Systems. ResearchCrux, v1.0. March 2026. https://github.com/CueCrux/ResearchCrux

CueCrux. The CROWN Receipt Protocol, v0.1. ResearchCrux. March 2026. https://github.com/CueCrux/ResearchCrux


License

CC BY 4.0. See LICENSE.

Key Links

About

Research, protocol specifications, and benchmark evidence for the CueCrux platform. CROWN receipt protocol, retrieval quality whitepapers, EU AI Act and DORA compliance mapping.

Topics

Resources

Contributing

Security policy

Stars

0 stars

Watchers

0 watching

Forks

Packages

Contributors

Languages