Skip to content

Fix filters after outer joins silently turning them into inner joins#2103

Merged
shangyian merged 4 commits into
DataJunction:mainfrom
shangyian:outer-joins
May 11, 2026
Merged

Fix filters after outer joins silently turning them into inner joins#2103
shangyian merged 4 commits into
DataJunction:mainfrom
shangyian:outer-joins

Conversation

@shangyian
Copy link
Copy Markdown
Collaborator

@shangyian shangyian commented May 9, 2026

Summary

When a v3 measures query joins to a dim via RIGHT OUTER / FULL OUTER, filters on adjacent left-joined dims land in the outer where after the outer join. This silently breaks the outer join's preservation semantics and preserved-side rows (the ones the outer join exists to keep) get dropped whenever the filter's column is NULL on them. This PR fixes that and consolidates the related filter-routing logic.

For queries that mix left join-with-filter and right/full outer, the result rows differ from before this PR: preserved-side rows from the outer join now survive when no matching fact row exists, matching the user's intent of right outer (preserve everything from the right). Queries that don't involve right/full are unaffected (pure-LEFT-chain filter narrowing semantics remain as today).

OUTER-JOIN-safe filter routing in CTE bodies

When a right or full outer join is present in the chain, every left/inner-joined dim that has a defeating filter is consolidated into a <parent>_filtered CTE that applies its filters before the outer join reaches it. The outer FROM then reads from the filtered CTE, and the outer join preserves correctly.

Absorbed-dim references (t2.client_name) are rewritten to the wrapper alias (t1.client_name) across projection, GROUP BY, kept-join ONs, and remaining WHERE atoms. The absorber handles both qualification styles DJ uses: _table-set (projection/GROUP BY) and name.namespace-set (filter atoms via _add_table_prefixes_to_filter).

Add COALESCE for outer join FKs

When the full-skip optimization substitutes a dim's PK with the fact's FK and a sibling dim shares the same FK alignment via its own link, the projection now emits COALESCE(t1.foreign_key, t3.primary_key) AS final_key and the GROUP BY mirrors the COALESCE. This preserves correct values under outer joins that null-fill the fact side.

Redundant outer-WHERE cleanup

The parent-alias atoms that are already pushed into the parent CTE (via the existing pushdown machinery) are no longer duplicated into the outer SELECT's WHERE. They were redundant and, under downstream OUTER JOINs, actively unsafe.

Test Plan

  • PR has an associated issue: #
  • make check passes
  • make test shows 100% unit test coverage

Deployment Plan

@netlify
Copy link
Copy Markdown

netlify Bot commented May 9, 2026

Deploy Preview for thriving-cassata-78ae72 canceled.

Name Link
🔨 Latest commit b60c624
🔍 Latest deploy log https://app.netlify.com/projects/thriving-cassata-78ae72/deploys/6a0232880d357f0008125766

@shangyian shangyian changed the title Fixed the v3 measures bug where filters on the non-preserved side of … Fix outer join + where push down May 9, 2026
@shangyian shangyian added the bug Something isn't working label May 9, 2026
@shangyian shangyian changed the title Fix outer join + where push down For v3, fix outer join + where push down May 9, 2026
@shangyian shangyian changed the title For v3, fix outer join + where push down filter routing in v3 measures SQL build May 10, 2026
@shangyian shangyian changed the title filter routing in v3 measures SQL build Fix WHERE-after-OUTER JOIN silently turning RIGHT/LEFT/FULL joins into INNER joins May 10, 2026
@shangyian shangyian changed the title Fix WHERE-after-OUTER JOIN silently turning RIGHT/LEFT/FULL joins into INNER joins Fix WHERE-after-OUTER-JOIN silently turning outer joins into inner joins May 10, 2026
@shangyian shangyian changed the title Fix WHERE-after-OUTER-JOIN silently turning outer joins into inner joins Fix filters after outer joins silently turning them into inner joins May 10, 2026
…an OUTER JOIN were being applied in WHERE, defeating the join.
@shangyian shangyian marked this pull request as ready for review May 11, 2026 20:13
@shangyian shangyian merged commit be75463 into DataJunction:main May 11, 2026
21 checks passed
@shangyian shangyian deleted the outer-joins branch May 11, 2026 20:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant