fix: reconciler must repair index in both directions

[guarzo]

The reconciler only computed stale = current ∖ fresh and deindexed those entries. The other direction — missing = fresh ∖ current — was silently ignored, so a lost SSE add_system event left the reverse index missing that entry permanently. The WebSocket subscription would never include the new system and killmails for it would be silently dropped until restart.

Observed in prod: subscribed_systems stayed at a stale value for 5+ hours across multiple reconciler ticks while real kills happened on systems that should have been re-added. Restart was the only recovery.

Make reconciliation symmetric — deindex stale AND index missing — so the hourly safety net actually does what its docstring claims. Also reduce the interval from 1 hour to 15 minutes; with two-way repair the worst-case recovery time is now bounded by the interval rather than the next restart.

Summary by CodeRabbit

• New Features

  • Reconciliation is now bidirectional—automatically adding missing systems as well as removing stale ones.

• Bug Fixes

  • Runs every 15 minutes for faster sync.
  • Safer handling of malformed upstream responses to avoid state corruption; indexing/deindexing proceed best-effort with warnings on individual failures.

• Tests

  • Expanded coverage for mixed stale/missing scenarios and continued reconciliation despite individual indexing failures.

• Documentation

  • Updated docs/spec to reflect new reconciliation semantics and failure modes.

• Chores

  • Project version bumped.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: ea59dcd7-f984-4f9c-a7d9-8978ee1775c8

📥 Commits

Reviewing files that changed from the base of the PR and between 44868b4 and 98b9d02.

📒 Files selected for processing (1)

• mix.exs

---

Walkthrough

Reconciler now performs two-way repair: deindexes local-only systems and indexes upstream-only systems. Reconcile interval changed from hourly to every 15 minutes. Malformed upstream responses that normalize to no ids return {:ok, :skipped_malformed} and preserve state. Added safe, best-effort indexing with warning logs.

Changes

Bidirectional Map Reconciliation

Layer / File(s)	Summary
Documentation & Configuration lib/wanderer_notifier/map/reconciler.ex	Moduledoc updated to describe two-way repair and malformed-response safety; reconcile interval changed from :timer.hours(1) to :timer.minutes(15); reconcile_map/2 @spec/docs updated to include {:ok, :skipped_malformed}.
Upstream Normalization & Guarding lib/wanderer_notifier/map/reconciler.ex	If a non-empty upstream list yields no normalized ids, reconciler logs a warning and returns {:ok, :skipped_malformed} preserving state instead of proceeding.
Core Reconciliation lib/wanderer_notifier/map/reconciler.ex	Replaced prune_stale_systems/3 with reconcile_systems/3 which computes stale_ids (local \ upstream) and missing_ids (upstream \ local) and logs drift counts.
Index/Deindex Execution lib/wanderer_notifier/map/reconciler.ex	Kept deindex_one_safe/3 best-effort semantics; added index_one_safe/3 and log_index_failure/3 so indexing missing ids proceeds best-effort and logs failures without aborting the pass.
Tests / Coverage test/wanderer_notifier/map/reconciler_test.exs	Added tests covering indexing missing upstream systems, combined stale+missing reconciliation, and index-error continuation; adjusted multi-map and deindex-error tests to stub index_system/2 where appropriate.
Manifest mix.exs	Project version bumped from 6.1.5 to 6.1.6.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  participant Reconciler as Reconciler
  participant Upstream as UpstreamAPI
  participant Local as LocalReverseIndex
  participant Registry as Registry

  Reconciler->>Upstream: fetch systems for map
  Upstream-->>Reconciler: list of system entries
  Reconciler->>Reconciler: normalize upstream ids
  alt normalized ids empty (malformed)
    Reconciler-->>Local: preserve state (skip) 
  else normalized ids present
    Reconciler->>Local: fetch local indexed ids for map
    Reconciler->>Reconciler: compute stale_ids, missing_ids
    alt stale_ids non-empty
      loop per stale_id
        Reconciler->>Registry: deindex_system(stale_id)
        Registry-->>Reconciler: ok / error
      end
    end
    alt missing_ids non-empty
      loop per missing_id
        Reconciler->>Registry: index_system(missing_id)
        Registry-->>Reconciler: ok / error
      end
    end
    Reconciler-->>Local: record reconciled state/logs
  end

Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

• guarzo/wanderer-notifier#116: Introduced the original Map.Reconciler and reverse-index pruning logic that this PR extends to bidirectional reconciliation.

Poem

I hopped through maps at quarter-past,
Found missing stars and cleared the last.
If upstream hides or local clings,
I add or prune with gentle springs.
Logs and warnings — tidy things. 🐇✨

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title directly summarizes the main change: implementing bidirectional repair in the reconciler to fix missing system entries, which is the core objective of this pull request.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/reconciler-bidirectional-repair

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@lib/wanderer_notifier/map/reconciler.ex`:
- Around line 42-44: The public contract of reconcile_map/2 is out of sync: the
implementation can return {:ok, :skipped_malformed} but the spec/docs only list
:reconciled | :skipped_empty; update the function spec/type and any `@doc/README`
mentions for reconcile_map/2 to include the new return {:ok, :skipped_malformed}
(or alternatively change the implementation to return one of the documented
atoms). Specifically, edit the `@spec` for reconcile_map/2 and its `@doc` examples
to declare returning {:ok, :reconciled} | {:ok, :skipped_empty} | {:ok,
:skipped_malformed} so Dialyzer is satisfied and documentation matches the code.
- Around line 278-286: The function prune_stale_systems/3 now performs two-way
reconciliation (deindexing stale IDs and indexing missing IDs) so rename it to a
name that reflects bidirectional behavior (e.g., reconcile_systems/3 or
sync_system_index/3) and update its private definition (defp prune_stale_systems
-> defp reconcile_systems) and all internal calls/usages in this module
(including the related logic around the block currently at 289-305) to the new
name to keep naming consistent and clear for future readers.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 7cbd065b-e93b-4a7c-8583-138ba70f2be2

📥 Commits

Reviewing files that changed from the base of the PR and between 3d030f6 and bc78cb0.

📒 Files selected for processing (2)

• lib/wanderer_notifier/map/reconciler.ex
• test/wanderer_notifier/map/reconciler_test.exs