Delivery traces and real entities: making the runtime visible in the UI

[lollox80]

Follow-up to #166, picking up two things you raised there: "something like the Traces for automations would be ideal for delivery selection" and "UI to do the simpler cases ... guard rails/guidance/feedback while still having full YAML for the advanced".

Both parts below are deliberately storage-neutral: neither adds config to the entry, neither writes YAML, so nothing here settles the round-trip decision for deliveries/scenarios in advance. They are about showing what the runtime already knows.

Checked against v2.2.1.

---

What this changes for the person using it

Both parts close the same gap: SuperNotify decides a great deal on every notification, and none of those decisions are visible or adjustable from the UI.

"Why didn't that arrive?" This evening my instance sent a lightning alert at high priority and Alexa stayed silent. The answer exists — high_priority wanted the announcement, a time-based DND scenario forbade it, false won — but to get it I had to patch the component, restart Home Assistant, call an undocumented action and read two hundred lines of YAML. What a user should see instead is one line per delivery:

Alexa — enabled by high_priority, disabled by dnd_schedule → not sent
Mobile push — sent to 4 devices (person.lorenzo → watch, iPad, phone, laptop)
Persistent — sent

That is Part A. The data behind those three lines already exists on every notification; it is thrown away.

"How do I silence this just for tonight?" Today there are two ways: edit YAML and restart, or hand-write a binary_sensor state in Developer Tools — a trick nobody would guess, and one the next restart forgets. With real entities it is a device page with a switch per delivery and per transport: usable from a dashboard card, usable inside automations ("guests arriving, turn voice announcements off"), and still set after a restart. Scenarios sit next to them as read-only lamps, so "is DND active right now?" is something you read rather than reconstruct. That is Part B.

What does not change: the YAML stays the source of truth. Those switches are runtime overrides, not config edits — like turning a radio down without changing its presets. Anyone with twenty-five scenarios keeps editing a file, which beats any form, exactly as you argued in #166.

The reason I think this pairing is worth your time: today SuperNotify gives people a very powerful engine and no dashboard. These two changes are the dashboard, and they are built almost entirely out of parts Home Assistant already provides — an action response, standard entities on a device — rather than out of frontend code living in this repo.

---

Part A — Delivery traces

What already exists

DebugTrace (model.py:784) is built for every notification (notification.py:115) and already records the whole selection story:

• delivery_selection per stage: scenario_enable_deliveries, scenario_disable_deliveries, default_enable_deliveries, recipient_enable_deliveries, override_disable_deliveries, override_enable_deliveries, ranked (notification.py:362-414)
• resolved: the target chain per delivery, 24 named stages from 100_delivery_default_fixed to 999_final_cut, with unchanged stages collapsed to NO_CHANGE (notification.py:757-859)
• delivery_artefacts and delivery_exceptions recorded by the transports (chime expanded targets, email html template failures, and so on)

That is, to my eye, already 80% of an automation-style trace. The cost is being paid on every notification.

Where it goes today

• It is only serialized through Notification.contents(diagnostics=True) (notification.py:634, debug_only = ["debug_trace"]), which happens only in the archive path, and only when the archive diagnostics outcome policy selects that notification's outcome — default ERROR (archive.py:104/138/191).
• enquire_last_notification calls contents() with no arguments (notify.py:226-227), so the trace is never in the action response.
• ATTR_DEBUG is accepted by the action schema (schema.py:560) and assigned to Notification.debug (notification.py:193), but that attribute is never read anywhere in the component. So debug: true on an action call currently does nothing.

Net effect: a successful-but-wrong delivery — the case people actually need to debug — leaves no trace anyone can reach without changing archive config and reading JSON off disk.

What it looks like on a live instance

I patched A0 (below) onto my own 2.2.1 and pulled a real trace. A lightning alert, priority: high, five scenarios active:

delivery_selection:
  scenario_enable_deliveries:  [alexa_announce, mobile_push, persistent]
  scenario_disable_deliveries: [alexa_announce, tts]
  default_enable_deliveries:   [alexa_announce, mobile_push, persistent]
  ranked:                      [mobile_push, persistent]
resolved:
  mobile_push:
    202_action_target:            {person_id: [person.lorenzo]}
    300_post_snooze:              NO_CHANGE
    310_resolve_indirect:         {person_id: [...], email: [...], mobile_app_id: [4 devices]}
    320_resolved_scenario_targets: NO_CHANGE
    330_delivery_selection:       {mobile_app_id: [4 devices]}
    610_delivery_split_targets:   NO_CHANGE
    620_narrow_to_direct:         NO_CHANGE
    999_final_cut:                NO_CHANGE

This is exactly the enabled conflict from point 3 of #166, caught in the wild: high_priority enables alexa_announce, notifiche_vocali_off and a time-based DND scenario disable it, false wins, and it silently drops out of ranked. The information needed to explain that is all there — but only if you diff two aggregate lists by eye.

Two limits worth fixing while the data is still cheap to change, both visible in that output:

1. The stage lists are aggregated, so provenance is lost. scenario_enable_deliveries and scenario_disable_deliveries do not say which scenario contributed each entry. The one thing a user wants to read — "alexa_announce: enabled by high_priority, disabled by cn_dnd_orario, result excluded" — cannot be reconstructed from the trace. Recording {delivery: [scenario, ...]} instead of a flat list is a small change to record_delivery_selection, and it is what makes a viewer explanatory rather than merely detailed.
2. resolved is empty for deliveries that never take targets. generate_targets returns early for TargetRequired.NEVER (notification.py:748), so a persistent delivery traces nothing at all. That is correct behaviour, but a viewer has to say "this transport does not use targets" rather than render an empty object — worth deciding at API level, not in the UI.

Proposal, in three separable steps

A0 — expose the existing trace on demand (tiny).
Add a trace: true field to enquire_last_notification, exactly like enquire_active_scenarios already does (notify.py:229-234), returning contents(diagnostics=True). About 20 lines plus a services.yaml field and a test. This alone makes "why did that go to the wrong place" answerable from Developer Tools today — the example above was produced this way, on a running 2.2.1 with ~20 deliveries and 25 scenarios, without touching the archive settings. Optionally, wire ATTR_DEBUG to force archiving with diagnostics for that one call, or drop the key if it is vestigial — your call.

A1 — a small ring buffer plus websocket API (core).
Keep the last N traces in memory (N configurable, default something like 20, no disk, dropped on restart), and add two websocket commands, supernotify/traces/list and supernotify/traces/get, admin-only. No frontend, no dependencies, and it is the same shape core uses for automation traces. Roughly 250-350 lines with tests.

A2 — the viewer itself, outside core.
I would build the cascade view as a Lovelace card in my own repo, not here: notification header, delivery selection per stage with what each stage added or removed, the target chain per delivery with NO_CHANGE collapsed, and exceptions inline. That keeps the frontend out of the integration entirely, consistent with how you have kept it so far.

Questions before I write anything beyond A0: do you want the websocket handlers in core at all, or would you rather the buffer be exposed only through an action response? And where should the buffer size live — options flow, YAML, or hardcoded?

---

Part B — Real entities for delivery / transport / scenario / recipient

What happens today

expose_entities() (notify.py:727) registers each item with entity_registry.async_get_or_create() and then writes its state with hass.states.async_set() (hass_api.py:180-184). These are registry entries with no platform entity behind them, which has a few concrete consequences:

• expose_entities() runs at initialization (notify.py:545), so states come back on restart — but at their configured value. A delivery a user turned off through the state write is silently on again after a restart, because there is nothing to restore. Registry entries for deliveries removed from config are never written again either, so they linger as permanently unavailable entities; my instance has a dozen.
• No device, so there is no single page in the UI that shows "what SuperNotify currently is": you have to know the entity_id naming convention to find anything.
• No unique_id semantics, no RestoreEntity, no translation keys, no icons.json state icons, and EntityCategory.DIAGNOSTIC only where it is passed explicitly.
• Enabling or disabling a delivery from the UI means writing a state by hand in Developer Tools, which _entity_state_change_listener (notify.py:625-690) then picks up. It works, and it is clever, but it is not something you can put in front of a non-technical user, and it does not survive a restart.

Proposal

Move these to real platforms on a single SuperNotify device:

• switch for deliveries and transports — the toggle users are already faking through state writes, with RestoreEntity so it survives restarts
• binary_sensor for scenarios — read-only, which is what feat: expose evaluated scenario state on scenario binary_sensors #171 makes meaningful (that PR is still open and rebased on main; it is the piece that gives these entities a real state instead of the hardcoded STATE_UNKNOWN at notify.py:741)
• sensor for notifications / failures, which are currently raw states.async_set calls (notify.py:588, notify.py:610)
• attributes stay as they are today, so existing dashboards keep working

The reason I think this is worth more than it looks: it is the cheapest possible "UI for the simple cases". A device page with toggles for deliveries and transports, plus scenario sensors, is generated by Home Assistant for free — no frontend code in this repo — and it is the guard-railed surface for the user who will never open YAML, while the YAML stays the source of truth for how those deliveries are defined. A toggle is a runtime override, not a config edit, so it does not collide with the round-trip model at all.

It also closes several quality_scale.yaml items currently marked todo: entity-category (partial today), entity-translations, icon-translations, entity-disabled-by-default, and docs-data-update, plus it gives diagnostics a natural home.

The part I would not decide alone

Deliveries and transports are binary_sensor.supernotify_* today. Turning them into switch.* changes entity ids and breaks anyone's automations that reference them. Options as I see them:

1. keep the binary_sensor entity ids as real platform entities and add switch entities alongside — no breakage, some duplication
2. migrate entity ids in the registry with a repair explaining the change — clean end state, one disruptive release
3. switches only for delivery and transport, binary sensors kept for scenario and recipient — smallest change that fixes the control path

I lean towards 3, but this is exactly the kind of trade-off you should pick.

---

Suggested order

A0 as a small PR straight away if you want it. #171 next, since it is already written and Part B leans on it. Then Part B behind whichever compatibility option you choose, and A1 only if you want the websocket API in core.

Happy to be told any of this is not wanted — I would rather ask first than send another prototype in the wrong direction.

[jeyrb]

I like this.

The blocker for making these full entities had been living in legacy notify shadow, not its a domain/platform of its own that should be simpler. Have also been thinking if more of these should be Notify Entities, however these are very limited, and only Recipient really fits well.

The binary_sensor is a bit backward, and should have been switches. Don't have any telemetry on usage for these, which is possibility though takes long time to be useful. Introducing the right thing, and having repairs for the old way is best, and then we can cut them out after 9-12 months

Is the ring buffer really needed? The traces will live in the archived JSON, so can be pulled back from there reasonably quickly when occasion demands.

[lollox80]

Thanks — that settles the parts I couldn't decide alone, so here's what I'd do with it.

Entities. Taking your call over my option 3: introduce switch for deliveries and transports as the real thing, keep the old binary_sensor ids working, and add a repair pointing at the replacement, with removal after a 9-12 month window. Agreed on Notify Entities too — Recipient is the only one where the semantics actually fit; forcing the others through it would cost more than it buys. Concretely, Part B becomes: switch (delivery, transport) with RestoreEntity, binary_sensor (scenario, recipient) as real platform entities, sensor for the two counters, all on one device, plus icons.json and translation keys — and a repair for the id change. That also picks up the entity-translations, icon-translations and entity-category items in quality_scale.yaml.

Ring buffer: you're right, drop A1. I checked what the archive actually gives you before answering. diagnostics is a full OutcomeSelection flag and settable from the options flow, so ALL already puts the trace in the JSON for every notification, not just errors — that covers the retrospective case without any new machinery, and the DebugTrace is built on every notification regardless, so nothing extra is being paid at runtime either way.

Two caveats worth knowing, neither of which needs a buffer:

• Nothing in the component reads the archive back — there's no action for it, so "pull it back from there" means going to the filesystem. That's fine for a maintainer, less so for the person asking "why didn't that arrive?" in the moment. A0 is exactly the missing step there: the trace comes back in the action response from Developer Tools, no archive reconfiguration, no disk.
• diagnostics: ALL is global and permanent, so every archived notification carries the full trace from then on. Fine when you're hunting something, heavier as a default on an instance with many deliveries.

So: A0 as a small PR, diagnostics: ALL documented as the way to keep traces after the fact, and A1 dropped. If you'd rather A0 didn't add a field to enquire_last_notification, the alternative is wiring ATTR_DEBUG — which is accepted by the action schema and assigned to Notification.debug but never read anywhere — to force diagnostics for that one call. Either is small; the field matches what enquire_active_scenarios already does, so that's the one I'd send unless you say otherwise.

The two provenance points from Part A still stand on their own, independently of where traces are read from: scenario_enable_deliveries / scenario_disable_deliveries are flat lists, so "enabled by X, disabled by Y" can't be reconstructed, and resolved is empty rather than explicit for TargetRequired.NEVER deliveries. Both are small changes in record_delivery_selection, and cheap to make now while nothing consumes that shape yet.

Order I'd suggest: A0 first, then #171 (already written, rebased, and Part B leans on it for scenario state), then Part B behind the repair.