Skip to content

fix(submit): batch daily_breakdowns writes on the merge path - #96

Closed
yoominho91 wants to merge 1 commit into
sculptdotfun:mainfrom
yoominho91:fix/batch-daily-upsert
Closed

fix(submit): batch daily_breakdowns writes on the merge path#96
yoominho91 wants to merge 1 commit into
sculptdotfun:mainfrom
yoominho91:fix/batch-daily-upsert

Conversation

@yoominho91

Copy link
Copy Markdown

Fixes the cause reported in #93.

What happens

updateExistingSubmission writes one daily_breakdowns row per incoming day, each as its own PostgREST call:

for (const day of data.ccData.daily) {
  ...
  if (prior) {
    await this.client.from("daily_breakdowns").update(dailyData)
      .eq("submission_id", existing.id).eq("date", day.date);
  } else {
    await this.client.from("daily_breakdowns").insert(dailyData);
  }
}

A submission with long history therefore costs one round-trip per day — about 270 in the report on #93. The route races that against a 25s timer, and the timer does not cancel the in-flight work, so the writes can land after the race has already rejected. The user gets a generic 500 telling them to check cc.json while their data is visible on the leaderboard, which invites retrying a write that already succeeded.

Change

daily_breakdowns has UNIQUE(submission_id, date) (001_initial_schema.sql:83), so one upsert with onConflict: "submission_id,date" covers both branches. The fold loop now collects rows and writes them in chunks of 500 afterwards, turning N round-trips into ceil(N/500) — 270 days becomes a single request.

Behaviour is otherwise unchanged: the per-machine merge in mergeMachineContribution still decides each row, so #43 semantics (new machine adds a slice, same machine replaces its own) are untouched. dailyMap is still populated the same way, so the totals recalculated below see the same values.

Scope

This removes the cause for the reported size rather than widening the timeout. The race itself — rejecting without cancelling, so a late success looks like a failure — is still there and worth handling separately; it just stops being reachable at these sizes. Happy to take that on in a follow-up if you want it.

Checks

  • pnpm exec tsc --noEmit: 14 pre-existing errors on main, still 14 after — none in the touched file.
  • pnpm build: compiles successfully. Page-data collection fails on both main and this branch without Supabase env vars, so unrelated.
  • Not exercised against a live database — I have no Supabase project for this repo, so the upsert path is unverified at runtime. Worth a check on your side before merging.

…otfun#93)

The merge path writes one row per incoming day with a separate PostgREST call,
so a submission with long history costs one round-trip per day — ~270 for the
report in sculptdotfun#93. The route races that against a 25s timer, and since the timer
doesn't cancel the in-flight work, the write can land after the race already
rejected. The user sees a generic 500 telling them to check cc.json while the
data is on the leaderboard, which invites a retry of a write that succeeded.

`daily_breakdowns` has UNIQUE(submission_id, date), so a single upsert with
`onConflict: "submission_id,date"` covers both the update and insert branches.
Collect the rows in the fold loop and write them in chunks of 500 afterwards,
turning N round-trips into ceil(N/500).

This removes the cause for the reported case rather than widening the timeout.
The race itself (rejecting without cancelling) is still there and worth handling
separately — it just stops being reachable at these sizes.
@vercel

vercel Bot commented Jul 31, 2026

Copy link
Copy Markdown

@yoo-minho is attempting to deploy a commit to the sculpt Team on Vercel.

A member of the Team first needs to authorize it.

@yoominho91

Copy link
Copy Markdown
Author

Closing — reopened from my personal account as #98 (same commit, same diff). This one was opened from a work account by mistake. Sorry for the noise.

@yoominho91 yoominho91 closed this Jul 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants