Add crash-durability journaling to Periscope (JournalKit)

AGENTS.md

@@ -8,7 +8,7 @@
 | SwiftFormat | 0.60.1   | `.mise.toml` |
 | Swift PM    | 6.2      | `Package.swift` (`swift-tools-version`) |
 
-**Libraries** (**StuffCore**, **LifecycleKit**, **LogKit**, **LogViewerUI**, **PeriscopeCore**, **PeriscopeUI**, **PeriscopeTools**, **TestHostSupport**, **RegionKit**, **WhereCore**, **WhereUI**) are defined in the root [`Package.swift`](Package.swift) — local package for libraries, Tuist for apps and test bundles.
+**Libraries** (**StuffCore**, **LifecycleKit**, **JournalKit**, **LogKit**, **LogViewerUI**, **PeriscopeCore**, **PeriscopeUI**, **PeriscopeTools**, **TestHostSupport**, **RegionKit**, **WhereCore**, **WhereUI**) are defined in the root [`Package.swift`](Package.swift) — local package for libraries, Tuist for apps and test bundles.
 
 Tuist manifests live at the repo root ([`Project.swift`](Project.swift), [`Tuist.swift`](Tuist.swift)). `Project.swift` references `Package.local(path: .relativeToRoot("."))` and declares the **Where** app, **StuffTestHost**, and unit-test targets that depend on package products.
 
@@ -53,8 +53,8 @@ by `./sync-agents`.
 
 ## Targets
 
-- **Package products** ([`Package.swift`](Package.swift)) — **StuffCore** ([`Shared/StuffCore/Sources/`](Shared/StuffCore/Sources/)), **LifecycleKit** ([`Shared/LifecycleKit/Sources/`](Shared/LifecycleKit/Sources/)), **LogKit** ([`Shared/LogKit/Sources/`](Shared/LogKit/Sources/), the logging facade), **LogViewerUI** ([`Shared/LogViewerUI/Sources/`](Shared/LogViewerUI/Sources/), the generic SwiftUI log viewer), **PeriscopeCore** / **PeriscopeUI** / **PeriscopeTools** ([`Shared/Periscope/`](Shared/Periscope/), the typed hierarchical observability framework — core + SwiftData store, SwiftUI environment integration, and on-device exploration tooling), **SwiftDataInspector** ([`Shared/SwiftDataInspector/Sources/`](Shared/SwiftDataInspector/Sources/), the generic SwiftData browser), and **TestHostSupport** ([`Shared/TestHostSupport/Sources/`](Shared/TestHostSupport/Sources/), the shared UIKit hosting + run-loop helpers for hosted test bundles) under [`Shared/`](Shared/); the **Broadway** design-system libraries **BroadwayCore** / **BroadwayUI** under [`Shared/Broadway/`](Shared/Broadway/) (see [`Shared/Broadway/AGENTS.md`](Shared/Broadway/AGENTS.md)); **RegionKit** ([`Where/RegionKit/Sources/`](Where/RegionKit/Sources/), the geometry + GeoJSON + region-lookup engine WhereCore builds on) / **WhereCore** / **WhereUI** under [`Where/`](Where/).
-- **Tuist targets** ([`Project.swift`](Project.swift)) — **Where** app ([`Where/Where/`](Where/Where/)) and its embedded app extensions **WhereWidgets** ([`Where/WhereWidgets/`](Where/WhereWidgets/), home/lock-screen widgets) and **WhereShareExtension** ([`Where/WhereShareExtension/`](Where/WhereShareExtension/), a Share-sheet action that saves shared content as evidence into the App Group store), **RegionViewer** ([`Where/RegionViewer/`](Where/RegionViewer/), a thin standalone **Mac Catalyst** host for the WhereUI region-map developer tool — the only target with a `.macCatalyst` destination), **BroadwayCatalog** ([`Shared/Broadway/BroadwayCatalog/`](Shared/Broadway/BroadwayCatalog/), the Broadway component showcase app), **StuffTestHost** ([`Shared/StuffTestHost/`](Shared/StuffTestHost/)), **WhereTests** (app tests, no host), and hosted **\*Tests** bundles (**StuffCoreTests**, **LifecycleKitTests**, **LogKitTests**, **LogViewerUITests**, **PeriscopeCoreTests**, **PeriscopeUITests**, **PeriscopeToolsTests**, **SwiftDataInspectorTests**, **RegionKitTests**, **WhereCoreTests**, **WhereUITests**, **BroadwayCoreTests**, **BroadwayUITests**, **BroadwayCatalogTests**) that depend on **StuffTestHost** + **TestHostSupport** + the relevant package product (most via the shared `unitTests` helper).
+- **Package products** ([`Package.swift`](Package.swift)) — **StuffCore** ([`Shared/StuffCore/Sources/`](Shared/StuffCore/Sources/)), **LifecycleKit** ([`Shared/LifecycleKit/Sources/`](Shared/LifecycleKit/Sources/)), **JournalKit** ([`Shared/JournalKit/Sources/`](Shared/JournalKit/Sources/), the generic append-only crash-durable journal), **LogKit** ([`Shared/LogKit/Sources/`](Shared/LogKit/Sources/), the logging facade), **LogViewerUI** ([`Shared/LogViewerUI/Sources/`](Shared/LogViewerUI/Sources/), the generic SwiftUI log viewer), **PeriscopeCore** / **PeriscopeUI** / **PeriscopeTools** ([`Shared/Periscope/`](Shared/Periscope/), the typed hierarchical observability framework — core + SwiftData store, SwiftUI environment integration, and on-device exploration tooling), **SwiftDataInspector** ([`Shared/SwiftDataInspector/Sources/`](Shared/SwiftDataInspector/Sources/), the generic SwiftData browser), and **TestHostSupport** ([`Shared/TestHostSupport/Sources/`](Shared/TestHostSupport/Sources/), the shared UIKit hosting + run-loop helpers for hosted test bundles) under [`Shared/`](Shared/); the **Broadway** design-system libraries **BroadwayCore** / **BroadwayUI** under [`Shared/Broadway/`](Shared/Broadway/) (see [`Shared/Broadway/AGENTS.md`](Shared/Broadway/AGENTS.md)); **RegionKit** ([`Where/RegionKit/Sources/`](Where/RegionKit/Sources/), the geometry + GeoJSON + region-lookup engine WhereCore builds on) / **WhereCore** / **WhereUI** under [`Where/`](Where/).
+- **Tuist targets** ([`Project.swift`](Project.swift)) — **Where** app ([`Where/Where/`](Where/Where/)) and its embedded app extensions **WhereWidgets** ([`Where/WhereWidgets/`](Where/WhereWidgets/), home/lock-screen widgets) and **WhereShareExtension** ([`Where/WhereShareExtension/`](Where/WhereShareExtension/), a Share-sheet action that saves shared content as evidence into the App Group store), **RegionViewer** ([`Where/RegionViewer/`](Where/RegionViewer/), a thin standalone **Mac Catalyst** host for the WhereUI region-map developer tool — the only target with a `.macCatalyst` destination), **BroadwayCatalog** ([`Shared/Broadway/BroadwayCatalog/`](Shared/Broadway/BroadwayCatalog/), the Broadway component showcase app), **StuffTestHost** ([`Shared/StuffTestHost/`](Shared/StuffTestHost/)), **WhereTests** (app tests, no host), and hosted **\*Tests** bundles (**StuffCoreTests**, **LifecycleKitTests**, **JournalKitTests**, **LogKitTests**, **LogViewerUITests**, **PeriscopeCoreTests**, **PeriscopeUITests**, **PeriscopeToolsTests**, **SwiftDataInspectorTests**, **RegionKitTests**, **WhereCoreTests**, **WhereUITests**, **BroadwayCoreTests**, **BroadwayUITests**, **BroadwayCatalogTests**) that depend on **StuffTestHost** + **TestHostSupport** + the relevant package product (most via the shared `unitTests` helper).
 - Add SPM library targets in `Package.swift` and wire apps/tests in `Project.swift` (see existing `unitTests` helper). A new module also ships a root `README.md` and `AGENTS.md` — see [Per-module docs](#per-module-docs).
 - **CI scheme**: CI runs the explicit shared **Stuff-iOS-Tests** scheme (all test bundles) rather than the autogenerated `Stuff-Workspace` scheme. New test bundles must be added to the `Stuff-iOS-Tests` scheme in `Project.swift` or CI won't run them.
 - **Never double-link a package product into a test bundle that already gets it through a dynamic-framework dependency.** Xcode's default SPM integration (how `Package.local` is wired here) embeds a product's code into *every* image that links it. **WhereUI** is a dynamic framework that statically embeds its own dependencies (WhereCore, BroadwayCore/BroadwayUI, LifecycleKit, LogViewerUI, SwiftDataInspector, …), so a `*Tests` bundle that depends on **WhereUI** *and* re-lists one of those in `extraPackageProducts` ends up with a **second copy** of it. When the shared **StuffTestHost** loads several `.xctest` bundles into one process, that leaves duplicate **type metadata** for the module, and any *type-keyed runtime lookup that crosses the WhereUI boundary* — SwiftUI `EnvironmentKey`s, `UITraitBridgedEnvironmentKey` bridging, the type-keyed `BTraits`/`BThemes`/`BStylesheets` containers — silently resolves against the wrong copy and returns the default (the writer stores under one copy's key *type*, the reader looks it up under another's). It only reproduces in the **full multi-bundle scheme** (not isolated `tuist test WhereUITests` runs) and is papered over by newer Xcode linkers, so it is brutal to diagnose (it cost ~2 hours once). **Depend on such products only transitively via `WhereUI`; keep them out of `extraPackageProducts`.** `WhereStylesheetTests.resolvesTraitAwareTokensFromTheBroadwayRoot` is the regression guard — it exercises a `\.stylesheet` (→ `\.bContext`) read across the boundary and fails if a duplicate copy returns.

Package.swift

@@ -10,6 +10,7 @@ let package = Package(
     products: [
         .library(name: "StuffCore", targets: ["StuffCore"]),
         .library(name: "LifecycleKit", targets: ["LifecycleKit"]),
+        .library(name: "JournalKit", targets: ["JournalKit"]),
         .library(name: "LogKit", targets: ["LogKit"]),
         .library(name: "LogViewerUI", targets: ["LogViewerUI"]),
         .library(name: "PeriscopeCore", targets: ["PeriscopeCore"]),
@@ -39,6 +40,10 @@ let package = Package(
                 .process("Resources"),
             ],
         ),
+        .target(
+            name: "JournalKit",
+            path: "Shared/JournalKit/Sources",
+        ),
         .target(
             name: "LogKit",
             path: "Shared/LogKit/Sources",
@@ -52,6 +57,9 @@ let package = Package(
         ),
         .target(
             name: "PeriscopeCore",
+            dependencies: [
+                .target(name: "JournalKit"),
+            ],
             path: "Shared/Periscope/PeriscopeCore/Sources",
         ),
         .target(

Project.swift

@@ -281,11 +281,18 @@ let project = Project(
             productDependency: "LogViewerUI",
             sources: ["Shared/LogViewerUI/Tests/**"],
         ),
+        unitTests(
+            name: "JournalKitTests",
+            bundleIdSuffix: "journalkit",
+            productDependency: "JournalKit",
+            sources: ["Shared/JournalKit/Tests/**"],
+        ),
         unitTests(
             name: "PeriscopeCoreTests",
             bundleIdSuffix: "periscopecore",
             productDependency: "PeriscopeCore",
             sources: ["Shared/Periscope/PeriscopeCore/Tests/**"],
+            extraPackageProducts: ["JournalKit"],
         ),
         unitTests(
             name: "PeriscopeUITests",
@@ -427,6 +434,7 @@ let project = Project(
                 "LifecycleKitTests",
                 "LogKitTests",
                 "LogViewerUITests",
+                "JournalKitTests",
                 "PeriscopeCoreTests",
                 "PeriscopeUITests",
                 "PeriscopeToolsTests",
@@ -446,6 +454,7 @@ let project = Project(
                 "LifecycleKitTests",
                 "LogKitTests",
                 "LogViewerUITests",
+                "JournalKitTests",
                 "PeriscopeCoreTests",
                 "PeriscopeUITests",
                 "PeriscopeToolsTests",
@@ -464,6 +473,7 @@ let project = Project(
         testScheme(name: "LifecycleKitTests"),
         testScheme(name: "LogKitTests"),
         testScheme(name: "LogViewerUITests"),
+        testScheme(name: "JournalKitTests"),
         testScheme(name: "PeriscopeCoreTests"),
         testScheme(name: "PeriscopeUITests"),
         testScheme(name: "PeriscopeToolsTests"),

Shared/JournalKit/AGENTS.md

@@ -0,0 +1,40 @@
+# JournalKit – Module Shape
+
+JournalKit is the generic append-only, crash-durable journal: synchronous
+`Data` appends that survive process death, segment rotation under a byte
+budget, and torn-tail-tolerant recovery. See [`README.md`](README.md) for
+the API and durability model.
+
+This file complements the root [`AGENTS.md`](../../AGENTS.md), which owns
+the build system, formatting, and global conventions. Read that first.
+
+## Scope & dependencies
+
+- **Foundation + os only.** No logging types, no Periscope imports — the
+  journal is payload-agnostic by design (PeriscopeCore layers log semantics
+  on top). Keep it that way.
+
+## Invariants
+
+- **`append` returning means the entry survives process death.** The write
+  reaches the kernel page cache synchronously; `.full` extends coverage to
+  kernel panics via `F_FULLFSYNC`. Nothing may buffer entries in user space.
+- **Recovery never throws over a torn tail.** A crash can cut the file at
+  any byte; recovery yields every wholly-written entry and flags the tear.
+  The truncation fuzz in `JournalRecoveryTests` pins this at every cut
+  point — keep it passing.
+- **A torn write poisons only its own segment.** A partial `write(2)`
+  (disk-full's shape) leaves bytes recovery stops at, so the segment is
+  marked poisoned and the next append rotates to a fresh one — later
+  entries must never land behind a tear.
+- **Drops are whole segments, oldest first,** and always observable
+  (`droppedSegmentCount`, `droppedOlderEntries`) — the newest entries are
+  never sacrificed. A segment that fails to delete stays in the byte
+  accounting (later rotations retry it) and the drop loop moves to the
+  next-oldest, so the budget still wins.
+
+## Testing
+
+Swift Testing in [`Tests/`](Tests), hosted in `StuffTestHost`
+(`JournalKitTests`). Tests journal into per-test temporary directories and
+construct crashed-journal states (truncation, corruption) directly on disk.

Shared/JournalKit/README.md

@@ -0,0 +1,64 @@
+# JournalKit
+
+An append-only, crash-durable journal of opaque `Data` entries — the
+write-ahead net for anything that must survive the process dying
+mid-flight. Periscope uses it as its log journal; the implementation is
+payload-agnostic and has no logging knowledge.
+
+## Quick start
+
+```swift
+import JournalKit
+
+// Writing (synchronous, any thread; ~microseconds per append):
+let journal = try Journal(
+    directory: journalDirectory,
+    configuration: Journal.Configuration(maximumByteCount: 8 * 1024 * 1024),
+)
+try journal.append(encodedEntry, sync: .processDeath)
+try journal.append(direEntry, sync: .full)   // F_FULLFSYNC — milliseconds
+
+// Recovery (the next launch):
+let recovered = try JournalRecovery.recover(directory: journalDirectory)
+for payload in recovered.payloads { ingest(payload) }
+try JournalRecovery.remove(directory: journalDirectory)
+```
+
+## Durability model
+
+- **`.processDeath`** (the default posture): once `append` returns, the
+  entry is in the kernel page cache and survives the *process* dying by any
+  means — crash, `SIGKILL`, jetsam. Cost: single-digit microseconds.
+  Verified empirically by the SIGKILL harness in
+  [`Shared/Periscope/Prototypes/JournalBenchmark`](../Periscope/Prototypes/JournalBenchmark).
+- **`.full`**: `F_FULLFSYNC` before returning — survives kernel panics and
+  power loss. Milliseconds; reserve for entries that warrant it.
+
+## How it works
+
+Entries are framed `[UInt32 length][UInt32 crc32][payload]` and appended to
+numbered segment files. Segments rotate at half the byte budget, and the
+oldest segment drops whole when the budget overflows — the journal favors
+the newest entries, like a flight recorder, and reports
+`droppedSegmentCount` / `droppedOlderEntries` so callers can surface the
+gap. An optional `segmentHeader` re-writes as the first entry of every
+segment, so identity/context entries survive any amount of rotation
+(recovery returns one copy per surviving segment). A *partial* write
+poisons its segment and the next append rotates, so torn bytes never
+strand later entries.
+
+Recovery reads segments in order and validates each entry's length and CRC.
+A torn or corrupt entry ends that segment's recovery and sets
+`foundTornEntry` — the partial final entry a crash leaves is expected, and
+recovery never throws over it.
+
+## Contracts & limitations
+
+- Entry ordering across concurrent appenders is append order (writers
+  serialize on an internal lock); per-writer order is preserved. Any
+  cross-entry semantics — sequencing, deduplication, schemas — belong to
+  the caller's payloads.
+- One `Journal` instance per directory at a time; reopening continues after
+  the existing segments rather than overwriting them.
+- CRC-32 catches torn and corrupt entries; it is integrity checking, not
+  authentication.