Skip to content

feat: v1.1 schema, 15 conformance fixtures, and corpus self-check - #3

Merged
sepehr-safari merged 5 commits into
mainfrom
feat/v1-schema-fixtures-conformance
Jul 22, 2026
Merged

feat: v1.1 schema, 15 conformance fixtures, and corpus self-check#3
sepehr-safari merged 5 commits into
mainfrom
feat/v1-schema-fixtures-conformance

Conversation

@sepehr-safari

Copy link
Copy Markdown
Collaborator

Seeds the repository with the three pieces promised in shiv3/ocpp-cp-simulator#188.

Schema

schema/trace-v1.schema.json is the v1.1 schema from the simulator's docs/trace-format.md, byte-identical apart from $id, which now points at this organization.

Fixtures

15 traces converted from the DebugKit scenario corpus (OCPP 1.6J, fully synthetic), each paired with an expected.json describing the consumer view a conformant implementation derives from it: correlation pairs, effective actions, unanswered calls, orphan responses.

Two properties worth knowing before review:

  • Responses omit the optional action field on purpose, so reproducing expected.json requires actually implementing messageId correlation.
  • raw is present on every record, and the self-check verifies it decodes to exactly the fields the record decomposes.

Coverage gaps are listed openly in fixtures/README.md: no CALLERROR records yet, no messageId reuse, single charge point, 1.6J only, no SOAP. @shiv3 the multi-CP and 2.0.1 traces you offered would close the biggest ones.

Conformance

conformance/README.md defines what conformance means for producers and consumers, including the correlation rule. conformance/validate.mjs is the corpus self-check and the reference consumer: schema validation, raw fidelity, and exact recomputation of every expected.json. npm test runs it, and CI runs it on every push and pull request, so a fixture and its expected view cannot drift apart unnoticed.

Notes

The JSON Schema from the v1.1 proof of concept, unchanged except for
$id, which now points at this organization's pages domain.

Signed-off-by: sepehr-safari <safari.sepehr@gmail.com>
…umer views

Each fixture pairs a trace (one record per line, raw present on every
record) with the consumer view a conformant implementation derives from
it: correlation pairs, effective actions, unanswered calls, and orphan
responses. Responses omit the optional action field on purpose, so
reproducing expected.json requires implementing correlation. All data is
synthetic. Known coverage gaps are listed in fixtures/README.md.

Signed-off-by: sepehr-safari <safari.sepehr@gmail.com>
conformance/README.md defines producer and consumer conformance and the
correlation rule the fixtures pin down. validate.mjs is the corpus
self-check and reference consumer: schema validation, raw fidelity, and
exact recomputation of every expected.json. npm test runs it; CI runs it
on every push and pull request.

Signed-off-by: sepehr-safari <safari.sepehr@gmail.com>

@shiv3 shiv3 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for turning this around so quickly — schema, corpus, and a runnable self-check in one go is exactly the seed this repo needed.

Verified the schema block is byte-identical to the simulator's docs/trace-format.md (only $id differs), and independently re-implemented the correlation algorithm (a different traversal than validate.mjs's backward scan) to cross-check all 15 expected.json files against their trace.jsonl — matches exactly, including the unresponsive-csms unanswered-call case. Solid corpus.

Two small things worth a look before/alongside the CALLERROR fixtures you flagged as a follow-up:

  1. checkRawFidelity() in validate.mjs only checks error.code against the raw frame for CALLERROR, not error.description/error.details — unlike the full-payload check it does for CALL/CALLRESULT. Worth completing now so the first CALLERROR fixture can't slip a partial mismatch.
  2. None of the 15 fixtures produce a non-empty orphanResponses, and that's not in fixtures/README.md's coverage-gap list (unlike CALLERROR, messageId reuse, etc.) — might be worth adding it there, or adding one small fixture that actually hits an orphan response, since it's one of the four consumer-view concepts the PR description leads with.

Minor/optional: the ok print in validate.mjs is gated on a global failures === 0, so on a red run later-but-passing fixtures stop printing their status line. Not a correctness issue, just a triage nit.

Approving — happy for the two items above to land here or as a follow-up, your call.

@sepehr-safari

Copy link
Copy Markdown
Collaborator Author

Thanks for the thorough pass, and especially for re-deriving the correlation with a different traversal and checking all 15 expected.json against it. Two independent implementations agreeing across the corpus is what this repo is for, so that cross-check is the part I care about most.

Both items are fair, and I'd rather close them here than defer:

  1. I'll extend checkRawFidelity() to compare error.description and error.details against the raw CALLERROR frame, matching what it already does for CALL/CALLRESULT, so the first CALLERROR fixture can't slip a partial mismatch.
  2. I'll add a small fixture that produces a real orphanResponses entry (a response with no preceding CALL), since it's one of the four consumer-view concepts and nothing currently exercises it, and keep the coverage-gap list honest.

I'll fold in the ok-print triage nit at the same time.

@juherr, this seeds the neutral repo so it's your call too. Happy to hold for your review, or to merge once those two fixtures are in if you're good with it.

… raw fidelity

Addresses shiv3's review on #3:
- checkRawFidelity() now checks error.description and error.details against
  the raw CALLERROR frame, matching the full-payload check for CALL/CALLRESULT,
  so the first CALLERROR fixture cannot slip a partial mismatch.
- A new orphan-response fixture produces a non-empty orphanResponses (a
  response whose CALL is not in the trace), the one consumer-view concept the
  corpus did not previously exercise.
- The per-fixture ok line is gated on that fixture's own result rather than the
  global failure count, so a red run no longer suppresses later passing lines.

Signed-off-by: sepehr-safari <safari.sepehr@gmail.com>
@sepehr-safari
sepehr-safari merged commit c9c2499 into main Jul 22, 2026
1 check passed
@sepehr-safari
sepehr-safari deleted the feat/v1-schema-fixtures-conformance branch July 22, 2026 14:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants