mark qrep failure in db and filter out of result - #4775
Conversation
ac3b86a to
81a6775
Compare
81a6775 to
dce2b56
Compare
❌ Test FailureAnalysis: Not flaky: the PR's new test Test_QRep_Retry_Sweeps_Unfinished_Run fails identically and instantly on all three matrix legs with a "duplicate key value violates unique constraint flows_name_unique" error, indicating the test/code creates the same QRep flow name twice; a separate pg18-only snapshot timeout is likely a genuine flake but is not the run-blocking failure. |
dce2b56 to
ac3b86a
Compare
Code reviewNo issues found. Checked for bugs and CLAUDE.md compliance. |
| ) (_ *protos.QRepFlowState, err error) { | ||
| defer func() { | ||
| if err != nil && !workflow.IsContinueAsNewError(err) && !temporal.IsCanceledError(err) && ctx.Err() == nil { | ||
| markQRepRunFailed(ctx, config, workflow.GetInfo(ctx).OriginalRunID) |
There was a problem hiding this comment.
This approach is generic but makes it a bit harder to match the misbehaving area with the error handling for it.
Wdyt of this approach:
- separate the first monitoring.InitializeQRepRun into its own activity right before GetQRepPartitions
- put this defer after it
There was a problem hiding this comment.
done, also separated out monitoring.InitializeQRepRun into RecordQRepRun and RecordQRepPartitions because the txn there for both tables was deceiving anyways as the second insert into qrep_runs is a no-op and that was always bothering me. Now it is more explicit.
When a QRep workflow fails (for example a snapshot clone during a CDC initial load or a table addition), its
peerdb_stats.qrep_runsrow keepsconsolidate_complete = false.InitialLoadSummaryreports it as in progress forever, and a retried attempt for the same table adds a second row, so the UI shows a table with more than one in-progress run.Changes:
failedboolean column topeerdb_stats.qrep_runsQRepFlowWorkflownow runs aMarkQRepRunFailedactivity when it returns an errorInitialLoadSummaryfilters out failed runs, so only completed and in-progress runs reach the UI.Testing:
Fixes: DBI-1105