Publish shaping: batching + deadband engine above the session - #3
Merged
Conversation
…he session The per-signal publish policy (mode, batchMs, deadband) now governs publication through a shaping engine (src/shaping.rs) that sits above the device session (ADP-5), shaping the mtconnect and sim backends identically: - batchMs=0 (default) publishes each reading immediately - unconfigured signals are untouched; batchMs>0 buffers GOOD readings per signal and flushes on window expiry as ONE SouthboundSignalUpdate whose samples[] carries the window's readings in arrival order, each keeping its own timestamps/extras. - mode: interval keeps only the latest reading per window; on-change keeps all. - BAD/UNCERTAIN readings flush their window immediately - a quality transition never sits in a window. - deadband (SAMPLE-category numerics, compiled against the probe model) applies on entry against the last accepted value; quality/qualityRaw changes, non-numeric values, and the first reading after connect/resync/resume pass. - Lifecycle: pause clears open windows (the resume snapshot republishes the current truth); shutdown/reconnect/link-loss flush them; repoll and the resume snapshot bypass shaping; reloads and model drift swap the policy table atomically with the signal-set swap, flushing changed windows on the old policy. One timer per instance task, virtual-clock testable. - defaults.publishMode now resolves at compile like defaults.batchMs and scopes to selection-derived SAMPLE signals. - New MtconnectAdapterShaping family (instance): published / coalesced / deadbandDropped pairs. - Docs describe the real behavior (configuration.md publish-shaping section, metrics.md, messaging-interface.md, schema descriptions); DESIGN.md records the decisions as D-MtconnectAdapter-L11. - Tests: shaping engine tables on a virtual clock, seam policy compilation, wire-shape + sim parity suite, and a live cppagent batching leg (run green against the compose harness).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The per-signal
publishconfig (mode,batchMs,deadband) now governs publication. A pure, virtual-clock-tested shaping engine (src/shaping.rs) sits above the device session (ADP-5), so themtconnectandsimbackends are shaped identically;src/supervisor.rsdrives it with one timer per instance task.Behavior
batchMs: 0publishes each reading immediately, exactly as before.batchMs > 0buffers GOOD readings per signal and flushes on window expiry as ONESouthboundSignalUpdatewhosesamples[]carries every buffered reading in arrival order — each sample keeping its ownserverTsand extras (sequence,receivedTs, …).mode: "interval"keeps only the latest reading per window.qualityRawchanges, non-numeric/array values, and the first reading after connect/resync/resume always pass.repolland the resume snapshot bypass shaping; shutdown/reconnect/link-loss flush open windows; a reload or model drift swaps the policy table atomically with the signal-set swap, flushing changed signals' windows with their old policy.defaults.publishModenow resolves at compile exactly likedefaults.batchMs(D-L10) and scopes to selection-derived SAMPLE signals.Observability
New
MtconnectAdapterShapingfamily (dims:instance):published/coalesced/deadbandDroppedTotal+Interval pairs. A new family rather than an extension ofMtconnectStream, whoseagentIddimension and shared-acquisition scope cannot carry a per-instance publication fact.Docs
configuration.mdpublish rows + a new "Publish shaping" section describe the real behavior;metrics.mddocuments the new family;messaging-interface.mddocuments multi-samplesamples[]; schema descriptions aligned;DESIGN.mdrecords the decisions as D-MtconnectAdapter-L11.Validation
cargo test: 362 passed, 0 failed (was 329 — all pre-existing tests unmodified exceptcompile_mtconnectsignature call sites).cargo clippy --all-targets -- -D warnings: clean.cargo llvm-cov --fail-under-lines 90(CI filter): 96.74% lines;shaping.rs99.77%.tests/compose.mtconnect-agent.yaml, cppagent 2.7): fullagent_integrationsuite 6/6 green, including the new live leg — three streamed values coalesced into one multi-sample update on the tiny device (EVIDENCE live shaping: buffered=[41.25, 42.25, 43.25] flushed_samples=4 (one update)).