Skip to content

Update SQL queries in GetIngestDefinition to use COALESCE for no records returned - #87

Merged
bennyaustin merged 1 commit into
mainfrom
Mirror-ELT-Pattern
Apr 26, 2026
Merged

Update SQL queries in GetIngestDefinition to use COALESCE for no records returned#87
bennyaustin merged 1 commit into
mainfrom
Mirror-ELT-Pattern

Conversation

@bennyaustin

Copy link
Copy Markdown
Owner

This pull request updates the GetIngestDefinition.sql stored procedure to improve the handling of NULL values in timestamp and number-based delta queries. The changes ensure that when no data is present in the source tables, the resulting queries return fallback values instead of NULLs, making downstream processing more robust.

Key improvements to NULL handling in generated SQL queries:

Delta Table Queries:

  • Updated all generated queries for delta tables to use COALESCE on MIN and MAX of the watermark columns, providing fallback values (such as the last known delta date or number) when no data is present. This prevents NULLs from being returned for DataFromTimestamp and DataToTimestamp. [1] [2] [3]

Running Number Queries:

  • Enhanced running number queries to use COALESCE for MIN and MAX values, defaulting to the last known delta number or calculated intervals, ensuring fallback values are always present. [1] [2]

These changes make the stored procedure more resilient by guaranteeing that summary queries always return valid timestamps or numbers, even when the source data is missing.…mTimestamp and DataToTimestamp calculations

@bennyaustin
bennyaustin merged commit 664f0a8 into main Apr 26, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant