fix(metadata): align GetCrossReference Thrift/SEA on parent identifiers - #645
fix(metadata): align GetCrossReference Thrift/SEA on parent identifiers#645eric-wang-1990 wants to merge 3 commits into
Conversation
GetCrossReference diverged between the Thrift and SEA protocols on the parent (PK-side) identifiers, and neither path matched the JDBC reference driver: 1. Thrift threw on an empty-string parent catalog. TGetCrossReferenceReq is null-guarded (a null parent is omitted, so the foreign-only GetColumnsExtended reuse works), but an empty-string parent is SENT as ParentCatalogName="", and the server rejects it (SHOW FOREIGN KEYS IN CATALOG `` -> TABLE_OR_VIEW_NOT_FOUND / INVALID_PARAMETER_VALUE). JDBC Thrift (DatabricksThriftServiceClient .listCrossReferences) catches isObjectNotFoundException and returns empty; ADBC Thrift had no such catch. Fix: catch object-not-found in DatabricksStatement.GetCrossReferenceAsync -> empty result. IsObjectNotFoundException gains a static overload over AdbcException so it matches the HiveServer2Exception the Thrift path throws. 2. SEA ignored the parent identifiers entirely. SHOW FOREIGN KEYS is scoped to the FOREIGN table, so it returns FKs to every parent; the JDBC reference filters the rows by the requested parent (CrossReferenceKeysDatabricksResultSetAdapter .includeRow) but ADBC SEA did not — over-returning for a table with FKs to multiple parents. Fix: filter returned rows by any specified (non-null) parent catalog/schema/table in GetCrossReferenceAsyncNoThrow. A null parent means "no constraint" (preserves the GetColumnsExtended foreign-only reuse, which passes null); an empty-string parent matches only an empty row value, so it filters to empty. Both paths now return an empty result for an empty-string parent + valid foreign side, matching the JDBC spec semantics, and SEA no longer over-returns for multi-parent tables. Closes the get_cross_reference Thrift-vs-SEA comparator diff without a whitelist (supersedes driver-test PR #1249). Unit tests: static IsObjectNotFoundException over HiveServer2Exception (empty-parent error shapes + unrelated-error negative), and ParentMatches (null=no-filter, exact + case-insensitive match, empty-string and non-matching filter out). 974 unit tests pass. Co-authored-by: Isaac
There was a problem hiding this comment.
Verdict: 1 Low
Looks good — a focused, well-tested Thrift/SEA parity fix for GetCrossReference. The static IsObjectNotFoundException(AdbcException) overload, the Thrift object-not-found catch, and the SEA ParentMatches filter all line up with the JDBC reference semantics, and the unit tests cover the key cases. One low-severity edge case noted inline around the null-parent-column fallback in the SEA filter.
Addresses: - #3762302992 at csharp/src/StatementExecution/StatementExecutionStatement.cs:1940 Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>
There was a problem hiding this comment.
Verdict: 2 Low
Looks good — a focused, correct alignment of GetCrossReference across Thrift/SEA that matches the JDBC reference (verified CrossReferenceKeysDatabricksResultSetAdapter.includeRow), with solid unit coverage of the two new helpers. Two Low notes: the Thrift catch broadens object-not-found swallowing to the FK side, and the new filter/catch wiring is only covered indirectly (predicate + classifier), not end-to-end.
Addresses: - #3762330882 at csharp/src/StatementExecution/StatementExecutionStatement.cs:1957 Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>
There was a problem hiding this comment.
Verdict: 1 Low
Looks good — a clean, JDBC-parity-aligned fix with strong unit coverage (predicate + end-to-end Http-seam tests). Verified the two GetCrossReferenceAsyncNoThrow call sites pass parents correctly (null for the GetColumnsExtended reuse, distinct parent/foreign options for the metadata path), that ShouldReturnEmptyPKFKResult does not short-circuit the empty-string-parent case, and that the static IsObjectNotFoundException overload + narrowly-guarded Thrift catch behave as described. One low note on null/absent server parent columns potentially over-filtering on SEA only.
Summary
GetCrossReferencediverged between the Thrift and SEA protocols on the parent (PK-side) identifiers, and neither path matched the JDBC reference driver. Surfaced by the ADBC C# Thrift-vs-SEA comparator (get_cross_referenceoutcome diff): with an empty-string parent catalog + a valid foreign side (fk_child), Thrift threw while SEA returned rows.Traced all four code paths for that input:
Root causes & fixes
1. Thrift threw on an empty-string parent catalog.
TGetCrossReferenceReqis null-guarded (HiveServer2Connection.cs) — a null parent is omitted (so the foreign-onlyGetColumnsExtendedreuse works), but an empty-string parent is sent asParentCatalogName="", which the server rejects (SHOW FOREIGN KEYS IN CATALOG \`→TABLE_OR_VIEW_NOT_FOUND/INVALID_PARAMETER_VALUE, verified live). JDBC Thrift (DatabricksThriftServiceClient.listCrossReferences) catchesisObjectNotFoundExceptionand returns empty; ADBC Thrift had no such catch. → Catch object-not-found inDatabricksStatement.GetCrossReferenceAsync→ empty result.IsObjectNotFoundExceptiongains a static overload overAdbcExceptionso it matches theHiveServer2Exceptionthe Thrift path throws (same pattern asIsDescTableExtendedUnsupported`).2. SEA ignored the parent identifiers entirely.
SHOW FOREIGN KEYSis scoped to the foreign table, returning FKs to every parent; the JDBC reference filters those rows by the requested parent (CrossReferenceKeysDatabricksResultSetAdapter.includeRow), but ADBC SEA did not — a latent over-return bug for any table with FKs to multiple parents.→ Filter returned rows by any specified (non-null) parent catalog/schema/table in
GetCrossReferenceAsyncNoThrow. A null parent means "no constraint" (preserves theGetColumnsExtendedforeign-only reuse, which passes null on all three); an empty-string parent matches only an empty row value → filters to empty.Both paths now return an empty result for an empty-string parent + valid foreign side (matching JDBC spec semantics — an empty-string identifier names no real object), and SEA no longer over-returns for multi-parent tables.
Test Plan
IsObjectNotFoundExceptionstatic overload overHiveServer2Exception— empty-parent error shapes (TABLE_OR_VIEW_NOT_FOUND/42P01,INVALID_PARAMETER_VALUE) return true; unrelated (ACCESS_DENIED) returns false.ParentMatches— null = no filter (matches any, incl. empty row); exact + case-insensitive match; empty-string and non-matching parent filter out.get_cross_referencediff collapses (supersedes driver-test PR #1249, which whitelisted it).Supersedes the whitelist approach in databricks-driver-test #1249 with a real driver fix.
This pull request and its description were written by Isaac.