Persist drawn ROIs into the layer's plot config

[SimonHeybrock]

Fixes #1090.

What changes for the user

Drawn ROIs stay drawn. Until now they lived only in the running plotter — and, on screen, only in the browser's edit tool — so anything that rebuilt either threw them away and republished the configured set over the top. Every route is fixed:

• restarting the dashboard
• removing and re-adding the plot layer
• reconfiguring the layer (changing colour, line width, max ROI count)
• restarting the workflow — the new job now gets the ROIs the user drew, not the ones in the config
• switching away from the grid and back, or any other re-render of the plot

The coordinates field tells the truth. It is rewritten as you draw, so opening "Reconfigure layer" shows the ROIs that are actually on the plot instead of a stale string from when the layer was created. It is labelled "Coordinates" rather than "Initial Coordinates" to match.

Coordinates are stored to six significant digits. A coordinate is a cursor pixel mapped onto the axis, so past the first few digits it is float noise — and since the stored string is also the field the user reads and edits, that noise was on screen. Rounding is applied to the stored value, not just the display: the field is editable, so a display that disagreed with the stored value would silently rewrite it on the next edit, and the round trip is now a fixed point, so a rebuild cannot perturb geometry. Significant digits rather than decimal places, because axes here run from pixel indices through Q in 1/Å to wavelength in m.

Typed-in coordinates are read off the axes. On a view with physical coordinates the field was interpreted as pixel indices, so typing [0,0,10,10] drew a box spanning 0–10 m on screen while telling the backend "pixels 0–10". Coordinates are now taken in the units the plot's axes show, which is what you get by reading numbers off the plot. Views with logical (unitless) coordinates are unaffected, and no in-repo instrument config seeds a request plotter's coordinates — but a layer saved on a production dashboard with typed-in coordinates on a geometric view will move to where the numbers actually pointed.

Implementation

Which of the two designs

The issue offered persist edits and seed from readback. Seeding from the readback turns out to be worse than it looks: ROI requests are addressed to JobId(source_name, job_number), so a new job's readback starts empty, and sync_job_states rebuilds the plotter on exactly that event. A readback-seeded plotter would therefore adopt the empty set and drop every drawn ROI on each workflow restart — trading a restart-scoped clobber for a per-run one. So: persist edits, dashboard stays authoritative.

Each accepted edit builds updated params and hands them to a persister injected by PlotOrchestrator, which stores them on the layer and writes the config store. It deliberately does not bump the topology version: the plotter and its presenters are the source of the change and must survive it.

Why the stored geometry carries no unit

Interval and PolygonROI read a missing unit as pixel indices, while edit-derived ROIs carry the unit of the readback's x/y coordinates, and geometric projections (xy_plane, cylinder_mantle) really do produce metres. Writing a drawn ROI back as bare numbers and reading it again would silently reinterpret metres as pixels, so the round trip forced the question.

The unit is now stamped on at parse time from the data and never stored: it belongs to the view a layer is bound to, so persisting it would create a value that goes stale as soon as the layer is repointed, plus rules to resolve the disagreement. A re-derived unit cannot disagree. The rationale is recorded in the roi_request_plots module docstring.

A consequence is that params can only be turned into ROIs once compute() has seen the units, so the initial set is seeded there rather than in __init__. Presenters are only ever created after compute() has produced cached state, so nothing observes the empty interval.

Keeping what is rendered and what is live in agreement

Drawn ROIs lived only in the browser's edit tool: a presenter sends its pipe once, in __init__, so the element the server holds stayed at the set that presenter was constructed with. Any re-render — switching back to a grid, a cell rebuilt on a job state change — rebuilds the edit tool from that stale element, and the browser syncs the difference back as an edit. That silently dropped ROIs before this PR; with the geometry persisted, the loss would have been written to disk and reapplied to the next workflow generation. The edit handler now returns the accepted set for the presenter to render, so the element cannot disagree with the plotter's live ROIs. The echo edit that pushing it may provoke is absorbed by the existing unchanged-edit guard.

Making the layout cheap enough to write per gesture

Persisting an edit rewrites the whole dashboard layout — every grid, cell and layer — and it happens on the loop that serves every session. That was fine while _persist_to_store was reached only by rare topology mutations; a drawn ROI reaches it once per mouse gesture, so it needed to get cheaper before this PR could land. Measured on a synthetic layout of image layers, per write:

layout	file	before	after
10 grids x 5 layers	42 KiB	44 ms	12 ms
10 grids x 10 layers	83 KiB	88 ms	24 ms
20 grids x 10 layers	166 KiB	203 ms	60 ms

Two changes get there. The store now emits with libyaml's CSafeDumper where the wheel provides it — the pure-Python emitter was over 99% of the cost, and the output is byte-identical, so a file written by either emitter is readable and diffable against the other. And _persist_to_store no longer writes: it marks the layout dirty and a debounced flush writes once per second, so a burst of mutations costs one write instead of one each. shutdown flushes before it tears the model down, and it is now reached in production: DashboardServices.stop stopped the update thread and the transport but left the orchestrator alone, so that flush ran only in tests and every restart dropped whatever the debounce still held. With no loop running (tests, screenshot runs) the write stays inline.

Net effect for the case that motivated this: dragging ROIs around now costs at most one ~12-24 ms write per second, instead of a ~44-88 ms write per gesture.

Known rough edge, not addressed here

The geometry is still stored as a string of bare numbers, because the params form builder renders no lists and no nested models — RectanglesCoordinates is documented as a wrapper existing "to get full-width card". The domain type RectangleROI already exists and already carries units; holding it directly would delete both parse() implementations, both _parse_initial_geometry, both _format_geometry and both validators. That needs the widget builder extended first, and a deliberate migration: invalid persisted params currently revert a layer's entire params to defaults with only a warning log.

Test plan

• Unit tests: edit written back into params, rebuilt plotter seeds from and republishes the persisted set, clearing all ROIs survives a rebuild, physical-unit round trip, polygon round trip, orchestrator persists rewritten params through a store reload.
• Unit tests: stored coordinates drop float noise and survive a small-magnitude axis, the round trip is a fixed point for rectangles and polygons, a degenerate stored rectangle is dropped rather than raised.
• Unit tests: a burst of mutations costs one store write, a later burst gets its own, shutdown writes the pending layout, and with no running loop the write is immediate; the file a store writes matches what the reference emitter would write.
• Manual: draw a rectangle on a detector view, restart the dashboard, confirm the ROI is still there and the backend spectrum does not change.
• Unit tests: the rendered element follows drawn rectangles and polygons, an edit that changes nothing does not re-render, and a polygon still being drawn does not reach the element.
• Unit tests: stop shuts the orchestrator down after the update thread is joined and before the transport stops.
• Manual: draw two rectangles, stop the workflow, switch grids and back, then restart — both survive and are republished to the new job.
• Manual: draw a rectangle, reconfigure the layer's colour, confirm the ROI survives and the modal showed the drawn coordinates.

[SimonHeybrock]

LGTM; color selection seems broken, but that was pre-existing