Start CSV book enrichment immediately after upload

[niklhut]

Important

Blocked by #279. Implement this only after the interactive core/enrichment split from #279 is available and stable.

Problem

CSV imports currently persist books and queue opted-in Open Library enrichment, but the normal enrichment path is a scheduled sweep that runs every five minutes.

Relevant code:

• server/services/library-transfer.service.ts
• server/services/book-enrichment.service.ts
• server/repositories/book-enrichment.repository.ts
• tasks/books/enrich-imported.ts
• app/pages/library/index.vue
• nuxt.config.ts

The existing worker is durable and includes claims, leases, retries, ISBN locking, additive metadata updates, and cleanup. The library page polls pending status every five seconds. However, a newly completed CSV upload may remain visibly incomplete until the next scheduled sweep starts, and the client-side update currently patches only coverPath and enrichment status rather than tags and richer metadata.

Dependency

This issue depends on #279 because the CSV path should reuse the same separation introduced there between:

• persisting a minimum viable core book;
• optional cover/metadata/tag enrichment;
• user-visible pending and terminal enrichment states;
• idempotent patching of enriched data into the UI.

Do not duplicate a second incompatible enrichment model inside the CSV import workflow.

Goal

After a CSV upload has persisted its core book records, start enrichment promptly and show progress/results in the UI without making the five-minute cron sweep the primary happy path.

Retain the durable job table and scheduled sweep as recovery for disconnects, failures, retries, and interrupted processing.

Proposed flow

1. Validate and persist the CSV import as it does today.
2. Create or update durable enrichment jobs for eligible imported records.
3. Return the import result once core persistence is complete; do not wait for all Open Library covers/tags.
4. Immediately start processing the imported batch through a client-held foreground request using the reusable enrichment service established in Return ISBN metadata immediately and enrich covers/tags incrementally #279.
5. Show per-book pending states in the library UI.
6. Apply cover, suggested/system tags, description, and other enriched fields to visible records as they become available.
7. Leave unfinished jobs eligible for retry and for the existing scheduled sweep.

The foreground request should identify the imported batch rather than accepting arbitrary book IDs, and the service must verify that the requesting user owns the batch/jobs it processes.

Transport

Prefer the transport established by #279:

• a normal authenticated foreground enrichment request returning patches; or
• an enrichment event stream if Return ISBN metadata immediately and enrich covers/tags incrementally #279 adopts streaming for independently arriving cover/tag updates.

The upload request itself should not remain open until the whole batch is enriched. Large imports need a quick import result followed by separately observable enrichment progress.

UI requirements

• Show a clear post-import summary immediately after core persistence.
• Books waiting for enrichment must show a spinner/skeleton for their covers rather than the terminal no-cover placeholder.
• Cover, tags, description, and other returned metadata should update without a full reload.
• Show useful terminal states for no_cover, not_found, and failed.
• Progress should survive navigation to/from the library page.
• The UI should communicate that the import succeeded even if optional enrichment is still running or later fails.

Cloudflare and reliability considerations

• Do not rely solely on ctx.waitUntil(); its post-response extension is limited to 30 seconds.
• Keep the immediate happy path attached to a client request.
• Preserve durable jobs so client disconnects do not lose work.
• Keep the five-minute scheduled task as a recovery sweep, not the expected start mechanism.
• Maintain bounded batch size and concurrency to respect Open Library pacing and Cloudflare/D1/R2 limits.
• Claims, leases, retries, and ISBN locks must remain safe under overlap between a foreground processor and the scheduled sweep.
• Processing remains at-least-once, so metadata and cover application must be idempotent.
• Cloudflare preview deployments intentionally omit cron triggers; the immediate foreground path should still work there.

References:

• Return ISBN metadata immediately and enrich covers/tags incrementally #279
• https://developers.cloudflare.com/workers/platform/limits/
• https://developers.cloudflare.com/workers/runtime-apis/context/
• https://openlibrary.org/developers/api?show_page_status=1

Data returned to the client

Extend the enrichment update contract beyond only coverPath and status. It should provide enough data to patch visible books consistently, including as applicable:

• user book ID and canonical book ID;
• cover path;
• enrichment status;
• suggested/system tags;
• description;
• publication metadata changed by enrichment;
• terminal error/outcome suitable for user-facing state.

Avoid repeatedly returning unchanged large fields if a compact versioned patch or updated-at cursor can provide the same result.

Concurrency and integrity

• Foreground processing and cron recovery must be able to race safely.
• Multiple imported records sharing an ISBN must not download/store duplicate covers unnecessarily.
• Existing manual/user metadata must not be overwritten.
• Removed or changed imported books must cause their jobs/claims to cancel safely.
• Downloaded but unreferenced covers must be cleaned up.
• A completed/no-cover/not-found job must not return to an infinite pending state.
• Retry backoff must remain bounded and observable.

Observability

Record structured timings and counts for:

• time from import completion to foreground enrichment start;
• claimed jobs;
• enriched/no-cover/not-found/retried/failed/cancelled jobs;
• Open Library request and pacing time;
• cover download and blob storage time;
• tag hydration time;
• fallback work later recovered by cron.

This should make it possible to confirm that normal imports begin enrichment immediately while the scheduled sweep is functioning only as recovery.

Acceptance criteria

• Return ISBN metadata immediately and enrich covers/tags incrementally #279 is complete and its staged enrichment primitives are reused rather than duplicated.
• CSV import returns after core persistence and does not wait for all cover/tag enrichment.
• Eligible enrichment begins immediately after upload through a client-held foreground request.
• Newly imported books show pending cover/enrichment state without waiting for the next five-minute sweep.
• Covers update automatically when stored.
• Suggested/system tags and richer metadata update automatically in visible library records.
• Import success remains distinct from optional enrichment success/failure.
• Disconnecting or navigating away does not lose durable work; unfinished jobs remain recoverable.
• The existing scheduled sweep remains enabled as recovery and can safely overlap foreground processing.
• Cloudflare preview works through the foreground path despite omitted cron triggers.
• Duplicate ISBNs and overlapping processors are idempotent and do not create duplicate books, tags, jobs, or blobs.
• Unit tests cover batch ownership, job claiming, overlap, retries, terminal states, and idempotency.
• API tests cover validation, authorization, batch scoping, and response/stream contracts.
• UI/E2E tests cover immediate post-import state, incremental updates, navigation, failures, and cron-independent operation.

Non-goals

• Reimplementing the interactive ISBN lookup work tracked in Return ISBN metadata immediately and enrich covers/tags incrementally #279.
• Removing the durable enrichment job table or recovery sweep.
• Making the CSV upload request wait for all enrichment.
• Adding WebSockets solely for import progress.
• Refactoring unrelated CSV parsing/export behavior.