Oracle-inferred SQL files panic sqruff parse (unregistered JSONObjectContentSegment); one file aborts whole scan

[tinovyatkin]

Summary

Any .sql file whose dialect resolves to Oracle panics deep inside sqruff during parsing, killing the entire mehen run — including directory scans where one such file aborts everything with zero results. The user never has to ask for Oracle: the inference heuristics (VARCHAR2, NVL(, DUAL, CONNECT BY, MINUS) select the dialect automatically.

Found while exercising SQL coverage mapping on PR #246 with real utPLSQL (Oracle PL/SQL) sources.

Reproduction

$ printf 'select nvl(a, 1) from dual;\n' > oracle_probe.sql
$ mehen metrics oracle_probe.sql --format json

thread 'main' panicked at .../sqruff-.../crates/lib-core/src/parser/grammar.rs:133:22:
Grammar refers to 'JSONObjectContentSegment' which was not found in the dialect.
$ echo $?
101

Blast radius — one bad file kills a whole scan (worker panics inside the ignore walker, run aborts, no output):

$ printf 'select 1;\n' > ok1.sql
$ printf 'select nvl(a, 1) from dual;\n' > bad.sql
$ mehen top-offenders . -M sql.loc.code

thread 'main' panicked at .../ignore-0.4.33/src/walk.rs:1532:31:
called `Result::unwrap()` on an `Err` value: Any { .. }
ERROR mehen_engine::top_offenders] worker error: a traversal worker panicked
$ echo $?
1

Root cause

sqruff v0.39.0 (our pin) ships an Oracle grammar that references a segment that is never registered in the dialect — crates/lib-dialects/src/oracle.rs:4831:

//     insert=[Ref("ListaggOverflowClauseSegment"), Ref("JSONObjectContentSegment")])
        Ref::new("JSONObjectContentSegment").to_matchable(),

(Ported from SQLFluff's Python dialect_oracle.py, where that segment exists.)

Two subtleties worth recording:

1. The panic is lazy — parse-time, not construction-time. sqruff_lib_dialects::kind_to_dialect(Oracle) succeeds; the unregistered Ref only resolves when the matcher first explores that grammar branch (AnyNumberOf::match_segments → prune_options → Ref::simple → OnceLock init → Dialect::ref panic). So dialect.rs's dialect_for_kind doc-comment ("this is the authoritative compiled-in check … it never panics") holds for construction but does not protect parsing. Any Oracle-parsed file containing a function call reaches the broken branch.
2. This is a known failure class in the SQLFluff lineage (Grammar refers to 'X' which was not found in the dialect — e.g. Grammar refers to 'UnboundedKeywordSegment' which was not found in the tsql dialect sqlfluff/sqlfluff#2673); in Python it raises a catchable RuntimeError per file, in sqruff it panics.

Upstream status

Already fixed: quarylabs/sqruff#3016 "fix(oracle): drop reference to undefined JSONObjectContentSegment", merged 2026-07-28 (0d0befd). First release containing it: v0.40.0 (2026-08-14) — verified via compare/v0.40.0...0d0befd → behind.

Proposed actions

1. Bump the sqruff pin v0.39.0 → v0.40.0 (needs the usual grammar/behavior diff review; 0.40 may shift other dialect behavior and metric snapshots).
2. Resilience independent of the pin: wrap the per-file sqruff parse in catch_unwind (in mehen-sql's analyze path, or the engine walker worker) and degrade to the existing lex-error/diagnostic path (sql.parser.* facts, sql.unparsable diagnostic) instead of aborting the run. We already degrade gracefully for unparsable segments; a panicking grammar should be the same class of event at run granularity. One quarantined file must not zero out a repository scan.
3. Regression guard: a test that parses a representative fixture under every compiled-in dialect (or at minimum every dialect reachable by inference: tsql, snowflake, postgres, bigquery, oracle, mysql), so a future pin bump with a broken grammar branch fails CI instead of production runs.

Workaround

Pin the dialect away from Oracle in-file (SQLFluff-parity directive, must be a bare directive line):

-- sqlfluff:dialect:ansi

Environment

• mehen 1.9.0, branch feat/coverage-category (also affects main — the pin predates the branch)
• sqruff tag=v0.39.0 (8396285)
• macOS (arm64), rustc 1.95

[tinovyatkin]

Seems like fixed at quarylabs/sqruff#3016, so we need as part of the work on this issue:

• upgrade sqlruff version and investigate why our auto deps upgrade is not working for it
• investigate and fix error reporting - this panic is not acceptable

[tinovyatkin]

Bump landed on the #246 branch as 759bac4:

• sqruff 0.39.0 → 0.40.0, switched from git-tag pins to =0.40.0 registry pins. That also answers the auto-upgrade question: Dependabot's cargo ecosystem only updates registry dependencies, so the sqruff-* group in dependabot.yml never matched the git pins. With registry pins the group becomes functional.
• Oracle probe (select nvl(a, 1) from dual;) and a real PL/SQL create or replace function now parse cleanly under the oracle dialect — zero unparsable segments, no panic.
• Added every_compiled_dialect_survives_function_call_parse: pins each compiled dialect via directive and parses function calls (the exact lazy-Ref path that panicked), so a future bump with a broken grammar branch fails CI.
• 0.40 grew DialectKind by five (exasol, greenplum→already present, hive, materialize, starrocks, teradata); hive is now directive-selectable since sparksql pulls its grammar — docs updated.

Remaining scope of this issue: panic-to-diagnostic error reporting — wrap the per-file sqruff parse in catch_unwind and degrade to the sql.unparsable diagnostic path so a poisoned file can never abort a scan, regardless of parser health.