feat: v1.1 schema, 15 conformance fixtures, and corpus self-check - #3
Conversation
The JSON Schema from the v1.1 proof of concept, unchanged except for $id, which now points at this organization's pages domain. Signed-off-by: sepehr-safari <safari.sepehr@gmail.com>
…umer views Each fixture pairs a trace (one record per line, raw present on every record) with the consumer view a conformant implementation derives from it: correlation pairs, effective actions, unanswered calls, and orphan responses. Responses omit the optional action field on purpose, so reproducing expected.json requires implementing correlation. All data is synthetic. Known coverage gaps are listed in fixtures/README.md. Signed-off-by: sepehr-safari <safari.sepehr@gmail.com>
conformance/README.md defines producer and consumer conformance and the correlation rule the fixtures pin down. validate.mjs is the corpus self-check and reference consumer: schema validation, raw fidelity, and exact recomputation of every expected.json. npm test runs it; CI runs it on every push and pull request. Signed-off-by: sepehr-safari <safari.sepehr@gmail.com>
shiv3
left a comment
There was a problem hiding this comment.
Thanks for turning this around so quickly — schema, corpus, and a runnable self-check in one go is exactly the seed this repo needed.
Verified the schema block is byte-identical to the simulator's docs/trace-format.md (only $id differs), and independently re-implemented the correlation algorithm (a different traversal than validate.mjs's backward scan) to cross-check all 15 expected.json files against their trace.jsonl — matches exactly, including the unresponsive-csms unanswered-call case. Solid corpus.
Two small things worth a look before/alongside the CALLERROR fixtures you flagged as a follow-up:
checkRawFidelity()invalidate.mjsonly checkserror.codeagainst the raw frame for CALLERROR, noterror.description/error.details— unlike the full-payload check it does for CALL/CALLRESULT. Worth completing now so the first CALLERROR fixture can't slip a partial mismatch.- None of the 15 fixtures produce a non-empty
orphanResponses, and that's not infixtures/README.md's coverage-gap list (unlike CALLERROR, messageId reuse, etc.) — might be worth adding it there, or adding one small fixture that actually hits an orphan response, since it's one of the four consumer-view concepts the PR description leads with.
Minor/optional: the ok print in validate.mjs is gated on a global failures === 0, so on a red run later-but-passing fixtures stop printing their status line. Not a correctness issue, just a triage nit.
Approving — happy for the two items above to land here or as a follow-up, your call.
|
Thanks for the thorough pass, and especially for re-deriving the correlation with a different traversal and checking all 15 Both items are fair, and I'd rather close them here than defer:
I'll fold in the @juherr, this seeds the neutral repo so it's your call too. Happy to hold for your review, or to merge once those two fixtures are in if you're good with it. |
… raw fidelity Addresses shiv3's review on #3: - checkRawFidelity() now checks error.description and error.details against the raw CALLERROR frame, matching the full-payload check for CALL/CALLRESULT, so the first CALLERROR fixture cannot slip a partial mismatch. - A new orphan-response fixture produces a non-empty orphanResponses (a response whose CALL is not in the trace), the one consumer-view concept the corpus did not previously exercise. - The per-fixture ok line is gated on that fixture's own result rather than the global failure count, so a red run no longer suppresses later passing lines. Signed-off-by: sepehr-safari <safari.sepehr@gmail.com>
…es-conformance # Conflicts: # README.md
Seeds the repository with the three pieces promised in shiv3/ocpp-cp-simulator#188.
Schema
schema/trace-v1.schema.jsonis the v1.1 schema from the simulator'sdocs/trace-format.md, byte-identical apart from$id, which now points at this organization.Fixtures
15 traces converted from the DebugKit scenario corpus (OCPP 1.6J, fully synthetic), each paired with an
expected.jsondescribing the consumer view a conformant implementation derives from it: correlation pairs, effective actions, unanswered calls, orphan responses.Two properties worth knowing before review:
actionfield on purpose, so reproducingexpected.jsonrequires actually implementing messageId correlation.rawis present on every record, and the self-check verifies it decodes to exactly the fields the record decomposes.Coverage gaps are listed openly in
fixtures/README.md: no CALLERROR records yet, no messageId reuse, single charge point, 1.6J only, no SOAP. @shiv3 the multi-CP and 2.0.1 traces you offered would close the biggest ones.Conformance
conformance/README.mddefines what conformance means for producers and consumers, including the correlation rule.conformance/validate.mjsis the corpus self-check and the reference consumer: schema validation,rawfidelity, and exact recomputation of everyexpected.json.npm testruns it, and CI runs it on every push and pull request, so a fixture and its expected view cannot drift apart unnoticed.Notes
docs/trace-format.md. Migrating it here is left to a follow-up so it lands under its author.