Skip to content

fix(metadata): align GetCrossReference Thrift/SEA on parent identifiers - #645

Open
eric-wang-1990 wants to merge 3 commits into
mainfrom
eric-wang/xref-empty-parent-fix
Open

fix(metadata): align GetCrossReference Thrift/SEA on parent identifiers#645
eric-wang-1990 wants to merge 3 commits into
mainfrom
eric-wang/xref-empty-parent-fix

Conversation

@eric-wang-1990

Copy link
Copy Markdown
Collaborator

Summary

GetCrossReference diverged between the Thrift and SEA protocols on the parent (PK-side) identifiers, and neither path matched the JDBC reference driver. Surfaced by the ADBC C# Thrift-vs-SEA comparator (get_cross_reference outcome diff): with an empty-string parent catalog + a valid foreign side (fk_child), Thrift threw while SEA returned rows.

Traced all four code paths for that input:

Behavior (before) JDBC reference
ADBC Thrift throws JDBC Thrift catches object-not-found → empty
ADBC SEA returns rows (ignores parent) JDBC SEA filters rows by parent → empty

Root causes & fixes

1. Thrift threw on an empty-string parent catalog. TGetCrossReferenceReq is null-guarded (HiveServer2Connection.cs) — a null parent is omitted (so the foreign-only GetColumnsExtended reuse works), but an empty-string parent is sent as ParentCatalogName="", which the server rejects (SHOW FOREIGN KEYS IN CATALOG \`TABLE_OR_VIEW_NOT_FOUND/INVALID_PARAMETER_VALUE, verified live). JDBC Thrift (DatabricksThriftServiceClient.listCrossReferences) catches isObjectNotFoundExceptionand returns empty; ADBC Thrift had no such catch. → Catch object-not-found inDatabricksStatement.GetCrossReferenceAsync→ empty result.IsObjectNotFoundExceptiongains a static overload overAdbcExceptionso it matches theHiveServer2Exceptionthe Thrift path throws (same pattern asIsDescTableExtendedUnsupported`).

2. SEA ignored the parent identifiers entirely. SHOW FOREIGN KEYS is scoped to the foreign table, returning FKs to every parent; the JDBC reference filters those rows by the requested parent (CrossReferenceKeysDatabricksResultSetAdapter.includeRow), but ADBC SEA did not — a latent over-return bug for any table with FKs to multiple parents.
→ Filter returned rows by any specified (non-null) parent catalog/schema/table in GetCrossReferenceAsyncNoThrow. A null parent means "no constraint" (preserves the GetColumnsExtended foreign-only reuse, which passes null on all three); an empty-string parent matches only an empty row value → filters to empty.

Both paths now return an empty result for an empty-string parent + valid foreign side (matching JDBC spec semantics — an empty-string identifier names no real object), and SEA no longer over-returns for multi-parent tables.

Test Plan

  • IsObjectNotFoundException static overload over HiveServer2Exception — empty-parent error shapes (TABLE_OR_VIEW_NOT_FOUND/42P01, INVALID_PARAMETER_VALUE) return true; unrelated (ACCESS_DENIED) returns false.
  • ParentMatches — null = no filter (matches any, incl. empty row); exact + case-insensitive match; empty-string and non-matching parent filter out.
  • Full unit suite: 974 passed, 0 failed, 0 skipped.
  • Comparator re-run confirms the get_cross_reference diff collapses (supersedes driver-test PR #1249, which whitelisted it).

Supersedes the whitelist approach in databricks-driver-test #1249 with a real driver fix.

This pull request and its description were written by Isaac.

GetCrossReference diverged between the Thrift and SEA protocols on the parent
(PK-side) identifiers, and neither path matched the JDBC reference driver:

1. Thrift threw on an empty-string parent catalog. TGetCrossReferenceReq is
   null-guarded (a null parent is omitted, so the foreign-only GetColumnsExtended
   reuse works), but an empty-string parent is SENT as ParentCatalogName="", and
   the server rejects it (SHOW FOREIGN KEYS IN CATALOG `` -> TABLE_OR_VIEW_NOT_FOUND
   / INVALID_PARAMETER_VALUE). JDBC Thrift (DatabricksThriftServiceClient
   .listCrossReferences) catches isObjectNotFoundException and returns empty; ADBC
   Thrift had no such catch. Fix: catch object-not-found in
   DatabricksStatement.GetCrossReferenceAsync -> empty result. IsObjectNotFoundException
   gains a static overload over AdbcException so it matches the HiveServer2Exception
   the Thrift path throws.

2. SEA ignored the parent identifiers entirely. SHOW FOREIGN KEYS is scoped to the
   FOREIGN table, so it returns FKs to every parent; the JDBC reference filters the
   rows by the requested parent (CrossReferenceKeysDatabricksResultSetAdapter
   .includeRow) but ADBC SEA did not — over-returning for a table with FKs to
   multiple parents. Fix: filter returned rows by any specified (non-null) parent
   catalog/schema/table in GetCrossReferenceAsyncNoThrow. A null parent means "no
   constraint" (preserves the GetColumnsExtended foreign-only reuse, which passes
   null); an empty-string parent matches only an empty row value, so it filters to
   empty.

Both paths now return an empty result for an empty-string parent + valid foreign
side, matching the JDBC spec semantics, and SEA no longer over-returns for
multi-parent tables. Closes the get_cross_reference Thrift-vs-SEA comparator diff
without a whitelist (supersedes driver-test PR #1249).

Unit tests: static IsObjectNotFoundException over HiveServer2Exception (empty-parent
error shapes + unrelated-error negative), and ParentMatches (null=no-filter, exact +
case-insensitive match, empty-string and non-matching filter out). 974 unit tests pass.

Co-authored-by: Isaac
@eric-wang-1990 eric-wang-1990 added the engineer-bot engineer-bot may fix this issue / take over this PR label Aug 11, 2026

@peco-review-bot peco-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: 1 Low

Looks good — a focused, well-tested Thrift/SEA parity fix for GetCrossReference. The static IsObjectNotFoundException(AdbcException) overload, the Thrift object-not-found catch, and the SEA ParentMatches filter all line up with the JDBC reference semantics, and the unit tests cover the key cases. One low-severity edge case noted inline around the null-parent-column fallback in the SEA filter.

Comment thread csharp/src/StatementExecution/StatementExecutionStatement.cs Outdated
Addresses:
  - #3762302992 at csharp/src/StatementExecution/StatementExecutionStatement.cs:1940

Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>

@peco-review-bot peco-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: 2 Low

Looks good — a focused, correct alignment of GetCrossReference across Thrift/SEA that matches the JDBC reference (verified CrossReferenceKeysDatabricksResultSetAdapter.includeRow), with solid unit coverage of the two new helpers. Two Low notes: the Thrift catch broadens object-not-found swallowing to the FK side, and the new filter/catch wiring is only covered indirectly (predicate + classifier), not end-to-end.

Comment thread csharp/src/DatabricksStatement.cs
Comment thread csharp/src/StatementExecution/StatementExecutionStatement.cs
Addresses:
  - #3762330882 at csharp/src/StatementExecution/StatementExecutionStatement.cs:1957

Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>

@peco-review-bot peco-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: 1 Low

Looks good — a clean, JDBC-parity-aligned fix with strong unit coverage (predicate + end-to-end Http-seam tests). Verified the two GetCrossReferenceAsyncNoThrow call sites pass parents correctly (null for the GetColumnsExtended reuse, distinct parent/foreign options for the metadata path), that ShouldReturnEmptyPKFKResult does not short-circuit the empty-string-parent case, and that the static IsObjectNotFoundException overload + narrowly-guarded Thrift catch behave as described. One low note on null/absent server parent columns potentially over-filtering on SEA only.

Comment thread csharp/src/StatementExecution/StatementExecutionStatement.cs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

engineer-bot engineer-bot may fix this issue / take over this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant