Skip to content

feat(sql): Phase 3 procedural SQL metrics - #257

Open
tinovyatkin wants to merge 26 commits into
mainfrom
feat/sql-procedural-metrics
Open

feat(sql): Phase 3 procedural SQL metrics#257
tinovyatkin wants to merge 26 commits into
mainfrom
feat/sql-procedural-metrics

Conversation

@tinovyatkin

Copy link
Copy Markdown
Contributor

Summary

Implements Phase 3 of the SQL metrics research foundation (§6.17, §12): the sql.procedural.* metric family for PL/SQL, T-SQL, MySQL, and BigQuery-scripting routines.

New metrics

  • Counts: routine_count, block_count, max_block_depth, if_count, loop_count, case_statement_count, exception_handler_count, return_count, raise_throw_count, dynamic_sql_count
  • Composites: sql.procedural.cyclomatic_complexity (Sonar's documented PL/SQL increments) and sql.procedural.cognitive_complexity (nesting-weighted, boolean-sequence rule) — both file-level and per-routine on Function spaces, both evidence-backed (metric == Σ contribution.amount by construction)
  • Embedded query attribution: sql.structural_complexity.max_embedded_query (§9.3) file-level; each routine's space carries its own embedded sql.structural_complexity
  • Dynamic SQL closes the documented Phase-1 change_risk deviation: +5 × dynamic_sql_count with a sql.change_risk.dynamic_sql reason code

Measurement model

One dialect-agnostic token state machine over procedural regions only (routine definitions, anonymous blocks, marker-gated Unparsable runs). Basis: empirical CST probes (recorded as parser comparison §9) showed Oracle's typed nodes, T-SQL's keyword-led statements, and unparsable MySQL/T-SQL bodies all share one classified token stream — comments/literals lex separately even inside Unparsable, so keyword scanning is trivia-safe, and one code path cannot double-count constructs that are both typed and keyword-visible. §9 also re-affirms sqruff over sqlparser v0.62 for Phases 3–4 (sqlparser hard-fails on CREATE PROCEDURE for both MsSql and Oracle dialects).

CASE-expression WHEN arms deliberately stay in the declarative sql.case.* family (documented deviation from Sonar's single-number model).

Statement classification

New anonymous_block kind (Oracle DECLARE…BEGIN…END, T-SQL IF/WHILE/BEGIN batch statements, BigQuery scripting). Unlike routine bodies, block bodies execute when the file is applied, so their DML/TCL now feeds sql.dml.*, object touches, and change risk; routine bodies stay excluded.

Leave-it-better fixes

  • Oracle DML was invisible: sqruff emits OracleUpdateStatement/OracleTableReference/… instead of the ANSI kinds; top-level Oracle DML classified as unknown and appeared in no sql.dml.*, object-touch, or change-risk metric. Dialect-folding SyntaxSets fix every scan.
  • sql.predicate.not_count no longer counts NOT NULL column constraints or IF NOT EXISTS guards (IS NOT NULL, NOT IN, NOT EXISTS still count).
  • Deleted dead PredicateFacts.in_count (computed, never published; IN folds into comparison_count per §6.7).
  • Dropped the unused dialect parameter threaded through facts::extract.

Docs

New schoolbook page docs/metrics/sql/procedural.mdx (worked cyclomatic example, measurement model, references); overview/roadmap updated — Phase 3 marked shipped with parser-bound limitations documented (PL/SQL cursor FOR loops and procedural CASE degrade to Unparsable, never mis-count).

Testing

  • 2 new golden fixtures (plsql_procedure_control_flow.sql, tsql_procedure_control_flow.sql) with hand-traced expected values asserted per family (PL/SQL: cyclomatic 12, cognitive 9, change risk 9; T-SQL through unparsable spill: cyclomatic 8, blocks 6) plus full-metric-map snapshots
  • Evidence-sum invariant tests for both procedural composites on both parse paths; benchmark profile skips evidence without changing metrics
  • Per-unit attribution tests incl. innermost-unit attribution for nested PL/SQL subprograms
  • Regression tests for Oracle DML classification, anonymous-block DML risk, and the not_count exclusions
  • Full workspace: cargo insta test --all-features --check --workspace --unreferenced reject --test-runner nextest1685/1685 passed, no unreferenced snapshots; cargo clippy --all-targets --all-features --locked clean; existing snapshot updates are purely additive (new zero-valued keys)

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant