proofs: DRAT refutations, Tseitin hard family, and a proof-shape bench - #67
Merged
Merged
Conversation
CDCL on an UNSAT instance emits a resolution refutation -- conflict analysis IS resolution and the learnt-clause sequence is the proof. This turns that from an incidental property into three usable things. DRAT emission + external oracle. `--cdcl --proof FILE` writes a DRAT refutation (one line per learnt clause plus the final empty clause), and tests/run_proof_check.sh has drat-trim verify it. This is the first external oracle the UNSAT path has ever had: a SAT answer self-verifies against its model, an UNSAT answer had nothing checking it, so a bug that over-pruned the search would report `s UNSATISFIABLE` with the whole suite green. Includes a planted-fault case -- lemmas removed from a valid proof, which the checker must reject -- because without it a misconfigured checker passes everything. Additions only, no `d` lines: withholding deletions leaves the checker a superset of our clauses, which can only make a RUP check succeed, never fail. Tseitin-on-torus generator. The existing xor_triangle case is the tractable regime (SAT, zero conflicts), so pigeonhole and complete-graph coloring were the only pressure -- and those are the same family, k(n+1) with n colors being pigeonhole relabelled. --proof-bench shows them producing byte-identical triples. tseitin_torus_case adds genuine parity + expansion hardness (Urquhart 1987), a different mechanism from pigeonhole's counting, so a change that helps both is real while one that helps only pigeonhole is exploiting counting structure. --proof-bench. Reports refutation size, clause space (new peak_learnts high-water counter) and depth (new max_level) per policy per family, with ladders and growth ratios. The axes trade off: at pigeonhole-6-5 geom-saved gives size 1276 / space 56 while geom-negative gives 1605 / 42, so size-only and space-only views pick different winners. Kept out of run_smoke.sh (seconds per solve) and run as its own CI step. Measured here: pigeonhole resolutions 39 -> 210 -> 1276 -> 12397 for n = 3..6 (x5.4, x6.1, x9.7, multiplier rising, as Haken 1985 requires). Tseitin 3x3 = 1974 resolutions; 4x4 = 95516 (x48 for one step of the expansion axis). Counters are byte-identical across runs, so counter claims need n=1 -- the n=5 rule is a wall-time rule. Also fixes the CLI running the scan/DPLL solver unconditionally and discarding the result whenever --watched/--persistent/--cdcl was given. On pigeonhole-6-5 (n=5 medians) that discarded solve was 531ms against 614ms of CDCL, so --cdcl went ~1145ms -> 614ms. Counters were never affected, so no previously reported measurement changes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CDCL on an UNSAT instance emits a resolution refutation — conflict analysis is resolution and the learnt-clause sequence is the proof. This turns that from an incidental property into three usable things.
1. DRAT emission + an external oracle for UNSAT
--cdcl --proof FILEwrites a DRAT refutation;tests/run_proof_check.shhasdrat-trimverify it.This is the first external oracle the UNSAT path has ever had. A SAT answer self-verifies against its model; an UNSAT answer had nothing checking it, so a bug that over-pruned the search would report
s UNSATISFIABLEand the whole suite would stay green.ddeletion lines — withholding deletions leaves the checker a superset of our clauses, which can only make a RUP check succeed, never fail. A proof that always verifies beats a smaller one that can spuriously fail.drat-trim's exit code, not a grep: it prefixess VERIFIEDwith a carriage return, so line-anchored patterns silently never match and such a check would pass unconditionally.2. Tseitin-on-torus — the hard XOR family
The existing
xor_triangle_caseis the tractable regime (SAT, closes with zero conflicts), which left pigeonhole and complete-graph coloring as the only pressure — and those are the same family. k(n+1) with n colors is pigeonhole relabelled, and--proof-benchshows them producing byte-identical size/space/depth triples.tseitin_torus_caseadds genuine parity + expansion hardness (Urquhart 1987), a different mechanism from pigeonhole's counting. A solver change that helps both is real; one that helps only pigeonhole is exploiting counting structure. The odd/even pair at the same size is the control: same graph, same 72 clauses, charge parity flipped, and the even case closes with zero conflicts.3.
--proof-bench— size / space / depthNew counters
peak_learnts(clause-space high-water mark, distinct from the survivingactive_learnts) andmax_level(deepest decision level). The three axes trade against each other, so a conflicts-only view hides real regressions — at pigeonhole-6-5,geom-savedgives size 1276 / space 56 whilegeom-negativegives 1605 / 42, and size-only vs space-only pick different winners.Kept out of
run_smoke.sh(seconds per solve on the hard families); runs as its own CI step.Measured on this box
Counters are byte-identical across runs, so counter claims need n=1 — the repo's n=5 rule is a wall-time rule, now stated as such in CLAUDE.md.
Also: CLI dispatch fix
The CLI ran the scan/DPLL solver unconditionally and discarded the result whenever
--watched,--persistent, or--cdclwas given. On pigeonhole-6-5 (n=5 medians) that discarded solve was 531ms against 614ms of CDCL, so--cdclwent ~1145ms → 614ms. Counters were never affected (the discarded result was overwritten), so no previously reported measurement changes.Verification
tests/run_smoke.shgreen, 3.4s → 4.4stests/run_proof_check.shgreen — 8 refutations verified, planted fault rejected, SAT case writes no proof--lintclean (only the pre-existing W021parse_argshint)Honest ceiling
Measurement bounds proof size from above. Every result that matters in proof complexity is a lower bound, and those are theorems. This instrument validates that a family behaves as theory predicts and falsifies "that family is easy" claims — it does not settle anything.
🤖 Generated with Claude Code