feat(watch): stream arriving mail as NDJSON over JMAP push - #65
Merged
Conversation
Closes #64. `fastmail watch` blocks and emits one JSON object per line as mail lands, so incoming email can drive a real-time shell loop rather than a cron job that re-lists the inbox and diffs it by hand. JMAP already carried the transport: the session advertises an `eventSourceUrl` (RFC 8620 §7.3) that we parsed and discarded. Push is treated purely as a wake-up — the `Email` state cursor lives in the CLI, and every notification, poll tick and reconnect runs `Email/changes` against it, so all three paths converge on the same answer and a lost notification costs latency rather than mail. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016ri9XAwMqKiHLrDyTuA3np
The watcher moves to `jmap::ArrivalWatcher` so the CLI command and the GraphQL subscription are two front ends on one implementation rather than two implementations that agree today. Cursor, backoff, reconnect and resync semantics are therefore identical by construction. Served at `/graphql/stream` over Server-Sent Events. SSE rather than WebSockets: the subscription is a server-to-client firehose, nothing is ever sent back up the socket, and SSE reconnects on its own. MCP gets nothing here on purpose — tools are request/response and a subscription never returns. The `graphql` tool's description now says so and points at the CLI and the HTTP surface instead. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016ri9XAwMqKiHLrDyTuA3np
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #64. Requested by David Celis (@davidcelis) by email — repo interaction limits are collaborators-only, so he couldn't file it himself.
What it does
fastmail watchblocks and emits one JSON object per line as mail arrives, so incoming email can drive a shell loop instead of a cron job that re-lists the inbox and diffs it by hand.--mailboxnarrows to one folder,--fullincludes bodies and attachment metadata,--poll <seconds>swaps the push connection for periodic checks.Same thing as a GraphQL subscription, when the HTTP surface is up:
Load-bearing decisions
One watcher, two front ends.
jmap::ArrivalWatcherholds the cursor, the push connection, the backoff and the resync policy;fastmail watchand theemailssubscription are both thin wrappers over it. They don't merely agree on semantics, they can't disagree.Push is only a wake-up, never the source of truth. JMAP has advertised the transport all along — the session carries an
eventSourceUrl(RFC 8620 §7.3) that we parsed intoSessionand discarded. But theEmailstate cursor lives in the CLI, and every notification, poll tick and reconnect runsEmail/changesagainst it. All three paths converge on the same answer, so a dropped connection or a missed frame costs latency rather than mail — and--pollis the same code with a timer instead of a socket, not a second implementation.Only creations are reported. Including updates would replay every flag change and folder move as an arrival, which isn't what a mail loop means by "new".
A dead credential ends the process; nothing else does. A watcher that exits on one bad response is useless in the loop it's meant to feed, so transient errors go to stderr and the loop continues. The exception is
cannotCalculateChanges— the server has discarded history past our cursor, there's no way to know what was missed, and replaying the mailbox as new would be a lie. It resyncs to now and says so on stderr. stdout stays pure NDJSON in every case.The push connection gets its own HTTP client. The shared one caps requests at 30s; a channel meant to stay open for days needs a read timeout instead, so silence reads as a dead connection rather than an idle one.
SSE rather than WebSockets for the subscription. It's a server-to-client firehose — nothing is ever sent back up the socket, and SSE reconnects on its own. It's also the same shape the CLI consumes from Fastmail, so there's one mental model for the whole path.
MCP gets nothing. Tools are request/response and a subscription never returns. The
graphqltool's description now says so explicitly and points at the CLI and/graphql/stream, rather than leaving a caller to discover it by hanging.Summaries by default —
Email/getwith bodies is the expensive half, and paying a document parse per arrival is the wrong default for a stream.Verifying
Tested live against Fastmail, both paths:
eventSourceUrlis present and templates as expected (https://ams.api.fastmail.com/jmap/event/?types={types}&closeafter={closeafter}&ping={ping}).fastmail watch --mailbox inbox→ sent mail to self → one NDJSON line on stdout within seconds, stderr silent.fastmail watch --mailbox inbox --poll 5→ same, via the timer path.POST /graphql/streamwithsubscription { emails(mailbox: "inbox") {...} }→ sent mail to self → arrived as an SSEdata:event with the selected fields.Automated coverage (211 tests green):
src/jmap/events.rs— SSE reassembly: frames split across chunks, split between\rand\n, repeateddata:lines, several frames per chunk, comment-only frames (Fastmail opens with one), missing space after the colon.src/jmap/mod.rs—email_changespaging viahasMoreChangeswith each page matched on the state it was asked for,cannotCalculateChangessurviving the trip throughparse_responseintact (the watcher keys its resync off it),email_statefetching state without mail,get_email_summariesnot requesting body values, andopen_event_streamfilling every placeholder in the URL template.src/jmap/watch.rs— mailbox membership filtering, which errors are fatal, and backoff growth to the ceiling (on a paused clock, so it asserts the timing without waiting it out).Note
GraphiQL isn't wired for subscriptions — its fetcher would need an SSE transport configured.
/graphql/streamis for programmatic consumers; the IDE still does queries and mutations as before.