Skip to content

Support LLVM IR tests with Clang 22 - #408

Draft
PhilippGrulich wants to merge 1 commit into
mainfrom
codex/check-and-regenerate-llvm-ir-golden-files
Draft

Support LLVM IR tests with Clang 22#408
PhilippGrulich wants to merge 1 commit into
mainfrom
codex/check-and-regenerate-llvm-ir-golden-files

Conversation

@PhilippGrulich

@PhilippGrulich PhilippGrulich commented Jul 17, 2026

Copy link
Copy Markdown
Member

Motivation

  • Clang 22 enables a -Wc2y-extensions warning which promoted Catch2's use of __COUNTER__ to an error under the project's -Werror policy and caused the LLVM IR test build to fail, so the build should suppress that warning for Clang 22+.

Description

  • Add a conditional in CMakeLists.txt to append -Wno-c2y-extensions to CMAKE_CXX_WARNINGS when CMAKE_CXX_COMPILER_VERSION is >= 22, keeping the rest of the project's warning/-Werror policy intact.

Testing

  • Configured and built with Clang 22 using cmake -S . -B build-clang22 -G Ninja -DCMAKE_C_COMPILER=clang-22 -DCMAKE_CXX_COMPILER=clang++-22 -DENABLE_TESTS=ON -DENABLE_LLVM_IR_TEST=ON, built nautilus-llvm-ir-test, and ran ctest --test-dir build-clang22/nautilus -R 'llvm-ir:' --output-on-failure, which completed with all tests passing (100% of run tests) and several tests skipped as expected; the LLVM IR golden files did not require regeneration.
  • Ran the formatting target cmake --build build-clang22 --target format successfully.

Codex Task

github-actions[bot]
github-actions Bot reviewed Jul 17, 2026
View reviewed changes

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tracing Benchmark

Details
Benchmark suite Current: 42a27d9 Previous: d1996fc Ratio
tiered_twotier_addOne 3.50072 ms (± 152.815) 3.73322 ms (± 117.817) 0.94
tiered_singletier_addOne 3.58856 ms (± 397.573) 3.59166 ms (± 133.343) 1.00
single_compile_mlir_addOne 3.40749 ms (± 116.663) 3.59131 ms (± 138.78) 0.95
e2e_tiered_bc_to_mlir 3552.6 us (± 146834) 3792.8700000000003 us (± 140743) 0.94
e2e_single_mlir 6.13521 ms (± 845.026) 6.2519 ms (± 195.358) 0.98
comp_mlir_add 6.09868 ms (± 379.249) 6.23705 ms (± 179.236) 0.98
comp_mlir_ifThenElse 6.70319 ms (± 246.849) 6.97285 ms (± 289.89) 0.96
comp_mlir_deeplyNestedIfElse 5.63695 ms (± 777.585) 5.65542 ms (± 155.032) 1.00
comp_mlir_loop 8.36222 ms (± 306.437) 7.89202 ms (± 205.921) 1.06
comp_mlir_ifInsideLoop 32.2614 ms (± 965.044) 31.0746 ms (± 518.566) 1.04
comp_mlir_loopDirectCall 13.2426 ms (± 199.963) 12.8414 ms (± 256.009) 1.03
comp_mlir_pointerLoop 31.4306 ms (± 1.1271) 30.2444 ms (± 974.359) 1.04
comp_mlir_staticLoop 5.50711 ms (± 230.783) 5.61296 ms (± 169.922) 0.98
comp_mlir_fibonacci 11.727 ms (± 378.127) 12.0296 ms (± 374.963) 0.97
comp_mlir_gcd 10.6219 ms (± 456.775) 10.9305 ms (± 285.625) 0.97
comp_mlir_nestedIf10 11.8039 ms (± 446.307) 12.0052 ms (± 566.594) 0.98
comp_mlir_nestedIf100 26.802 ms (± 1.27816) 27.395 ms (± 520.802) 0.98
comp_mlir_chainedIf10 10.3685 ms (± 366.339) 11.2748 ms (± 403.055) 0.92
comp_mlir_chainedIf100 22.0601 ms (± 539.781) 23.3786 ms (± 325.589) 0.94
comp_cpp_add 26.9063 ms (± 1.1071) 29.2402 ms (± 662.069) 0.92
comp_cpp_ifThenElse 27.7671 ms (± 696.997) 29.3094 ms (± 377.308) 0.95
comp_cpp_deeplyNestedIfElse 28.285 ms (± 459.832) 30.0998 ms (± 962.754) 0.94
comp_cpp_loop 27.7267 ms (± 654.53) 29.2341 ms (± 365.465) 0.95
comp_cpp_ifInsideLoop 28.9599 ms (± 412.295) 29.6279 ms (± 660.025) 0.98
comp_cpp_loopDirectCall 28.7267 ms (± 372.927) 29.3826 ms (± 463.568) 0.98
comp_cpp_pointerLoop 28.6885 ms (± 447.008) 29.7897 ms (± 496.445) 0.96
comp_cpp_staticLoop 28.7822 ms (± 1.56093) 29.1664 ms (± 444.455) 0.99
comp_cpp_fibonacci 28.7304 ms (± 706.436) 29.6257 ms (± 801.915) 0.97
comp_cpp_gcd 28.5084 ms (± 560.256) 29.5757 ms (± 2.3177) 0.96
comp_cpp_nestedIf10 30.3459 ms (± 616.051) 31.7323 ms (± 776.739) 0.96
comp_cpp_nestedIf100 56.1651 ms (± 1.13981) 57.1167 ms (± 4.30433) 0.98
comp_cpp_chainedIf10 31.2983 ms (± 888.045) 31.4501 ms (± 1.24862) 1.00
comp_cpp_chainedIf100 61.4346 ms (± 883.671) 60.4016 ms (± 3.16531) 1.02
comp_bc_add 15.8885 us (± 3.0178) 14.9058 us (± 2.86027) 1.07
comp_bc_ifThenElse 21.2658 us (± 4.75467) 17.4103 us (± 4.32805) 1.22
comp_bc_deeplyNestedIfElse 26.6989 us (± 4.95084) 22.476 us (± 4.68026) 1.19
comp_bc_loop 21.4664 us (± 4.67407) 18.0413 us (± 4.12339) 1.19
comp_bc_ifInsideLoop 25.558 us (± 5.02258) 21.0503 us (± 4.45333) 1.21
comp_bc_loopDirectCall 21.7083 us (± 3.73968) 18.2467 us (± 4.61525) 1.19
comp_bc_pointerLoop 24.639 us (± 6.73332) 19.5567 us (± 5.38957) 1.26
comp_bc_staticLoop 19.4638 us (± 4.21304) 17.7089 us (± 3.93654) 1.10
comp_bc_fibonacci 21.1658 us (± 3.63718) 18.2727 us (± 4.08081) 1.16
comp_bc_gcd 20.7264 us (± 3.94526) 17.54 us (± 3.99097) 1.18
comp_bc_nestedIf10 53.1569 us (± 11.8813) 41.7653 us (± 9.45988) 1.27
comp_bc_nestedIf100 289.75 us (± 17.1899) 281.457 us (± 9.72606) 1.03
comp_bc_chainedIf10 66.8903 us (± 11.0513) 55.1012 us (± 10.2569) 1.21
comp_bc_chainedIf100 487.102 us (± 22.4598) 457.243 us (± 13.5843) 1.07
comp_tbc_add 2.04313 us (± 244.335) 1.79091 us (± 123.167) 1.14
comp_tbc_ifThenElse 4.34735 us (± 617.591) 3.94704 us (± 353.996) 1.10
comp_tbc_deeplyNestedIfElse 9.88366 us (± 1.43064) 9.08337 us (± 1.40325) 1.09
comp_tbc_loop 4.44089 us (± 539.107) 3.96725 us (± 327.491) 1.12
comp_tbc_ifInsideLoop 7.62416 us (± 1.08982) 6.8764 us (± 788.217) 1.11
comp_tbc_loopDirectCall 6.50598 us (± 3.17029) 4.22833 us (± 356.378) 1.54
comp_tbc_pointerLoop 5.7763 us (± 894.989) 5.21685 us (± 572.675) 1.11
comp_tbc_staticLoop 4.75792 us (± 520.789) 4.32236 us (± 396.935) 1.10
comp_tbc_fibonacci 4.76757 us (± 425.808) 4.35932 us (± 406.586) 1.09
comp_tbc_gcd 3.94773 us (± 457.638) 3.52657 us (± 272.621) 1.12
comp_tbc_nestedIf10 26.5498 us (± 3.01972) 24.8382 us (± 2.62322) 1.07
comp_tbc_nestedIf100 282.029 us (± 11.0432) 278.14 us (± 7.11991) 1.01
comp_tbc_chainedIf10 34.6206 us (± 4.40984) 31.5848 us (± 3.16631) 1.10
comp_tbc_chainedIf100 411.944 us (± 22.1437) 404.876 us (± 11.3046) 1.02
comp_asmjit_add 25.0309 us (± 5.46768) 18.9113 us (± 5.39223) 1.32
comp_asmjit_ifThenElse 34.4963 us (± 5.73675) 25.2387 us (± 5.89852) 1.37
comp_asmjit_deeplyNestedIfElse 52.3251 us (± 9.53552) 35.3326 us (± 6.83989) 1.48
comp_asmjit_loop 34.4048 us (± 5.24748) 25.452 us (± 6.1411) 1.35
comp_asmjit_ifInsideLoop 50.6481 us (± 11.7019) 33.2754 us (± 6.42552) 1.52
comp_asmjit_loopDirectCall 45.9114 us (± 8.72542) 28.5771 us (± 6.22632) 1.61
comp_asmjit_pointerLoop 46.8645 us (± 8.72798) 30.4931 us (± 6.309) 1.54
comp_asmjit_staticLoop 30.3844 us (± 5.33824) 24.2662 us (± 5.80875) 1.25
comp_asmjit_fibonacci 36.475 us (± 5.68143) 27.0521 us (± 6.19035) 1.35
comp_asmjit_gcd 36.4173 us (± 6.40877) 26.186 us (± 6.26171) 1.39
comp_asmjit_nestedIf10 79.9493 us (± 12.0295) 68.0712 us (± 12.7893) 1.17
comp_asmjit_nestedIf100 507.969 us (± 22.4033) 499.14 us (± 16.2666) 1.02
comp_asmjit_chainedIf10 98.9171 us (± 13.9279) 89.7495 us (± 13.5101) 1.10
comp_asmjit_chainedIf100 685.92 us (± 89.7729) 670.929 us (± 17.1005) 1.02
trace_add 2.58135 us (± 421.709) 2.48514 us (± 286.374) 1.04
completing_trace_add 2.68223 us (± 454.753) 2.46315 us (± 214.257) 1.09
trace_ifThenElse 9.6338 us (± 1.43269) 8.80811 us (± 1.97333) 1.09
completing_trace_ifThenElse 5.06744 us (± 935.835) 4.70249 us (± 551.876) 1.08
trace_deeplyNestedIfElse 28.6364 us (± 3.89485) 25.8443 us (± 3.5215) 1.11
completing_trace_deeplyNestedIfElse 13.8356 us (± 2.35189) 12.9573 us (± 1.74989) 1.07
trace_loop 9.31164 us (± 1.41836) 8.71974 us (± 1.34599) 1.07
completing_trace_loop 5.25138 us (± 921.402) 4.81737 us (± 533.404) 1.09
trace_ifInsideLoop 18.7379 us (± 3.4197) 16.7956 us (± 3.42609) 1.12
completing_trace_ifInsideLoop 9.2783 us (± 1.49133) 8.70481 us (± 1.48558) 1.07
trace_loopDirectCall 9.88076 us (± 1.76308) 8.91554 us (± 1.52239) 1.11
completing_trace_loopDirectCall 5.44106 us (± 913.401) 4.88523 us (± 624.798) 1.11
trace_pointerLoop 15.75 us (± 3.96974) 14.5568 us (± 2.43507) 1.08
completing_trace_pointerLoop 11.4891 us (± 2.2567) 10.4043 us (± 1.4187) 1.10
trace_staticLoop 8.11789 us (± 1.03909) 8.11469 us (± 919.884) 1.00
completing_trace_staticLoop 8.44745 us (± 1.31731) 8.12886 us (± 1.13245) 1.04
trace_fibonacci 11.4119 us (± 2.02694) 10.4361 us (± 1.75704) 1.09
completing_trace_fibonacci 6.41845 us (± 857.408) 6.20322 us (± 777.911) 1.03
trace_gcd 12.7032 us (± 3.40419) 7.82946 us (± 1.31486) 1.62
completing_trace_gcd 4.4707 us (± 744.789) 4.06174 us (± 417.598) 1.10
trace_nestedIf10 48.5531 us (± 10.7758) 46.5369 us (± 7.35354) 1.04
completing_trace_nestedIf10 47.5181 us (± 8.17036) 46.1069 us (± 8.1331) 1.03
trace_nestedIf100 2.01145 ms (± 54.0637) 2.00986 ms (± 33.7633) 1.00
completing_trace_nestedIf100 1.98733 ms (± 53.831) 2.02558 ms (± 41.9295) 0.98
trace_chainedIf10 108.446 us (± 10.8088) 102.873 us (± 9.62448) 1.05
completing_trace_chainedIf10 52.4614 us (± 10.071) 51.171 us (± 7.37136) 1.03
trace_chainedIf100 5.14922 ms (± 92.7805) 4.97353 ms (± 106.162) 1.04
completing_trace_chainedIf100 2.36798 ms (± 57.9653) 2.36393 ms (± 31.103) 1.00
ssa_add 185.088 ns (± 23.516) 165.701 ns (± 8.69882) 1.12
ssa_ifThenElse 471.087 ns (± 63.9752) 413.574 ns (± 22.9547) 1.14
ssa_deeplyNestedIfElse 1.20164 us (± 144.844) 1.03483 us (± 56.4243) 1.16
ssa_loop 521.209 ns (± 80.3516) 437.486 ns (± 28.0728) 1.19
ssa_ifInsideLoop 949.806 ns (± 115.952) 785.881 ns (± 60.7208) 1.21
ssa_loopDirectCall 524.436 ns (± 77.9568) 439.568 ns (± 28.5778) 1.19
ssa_pointerLoop 591.134 ns (± 49.0646) 522.771 ns (± 28.2523) 1.13
ssa_staticLoop 429.061 ns (± 58.7476) 394.651 ns (± 19.2082) 1.09
ssa_fibonacci 529.816 ns (± 77.4928) 460.122 ns (± 43.9982) 1.15
ssa_gcd 487.949 ns (± 69.8049) 418.224 ns (± 42.886) 1.17
exec_mlir_add 11.531 ns (± 1.38256) 10.5599 ns (± 1.33058) 1.09
exec_mlir_fibonacci 14.8072 us (± 1.9363) 15.7129 us (± 2.04099) 0.94
exec_mlir_sum 606.173 us (± 33.6095) 551.558 us (± 60.9435) 1.10
exec_cpp_add 4.87051 ns (± 0.778922) 4.49947 ns (± 0.252743) 1.08
exec_cpp_fibonacci 49.6617 us (± 6.47077) 54.9282 us (± 4.89188) 0.90
exec_cpp_sum 11.4462 ms (± 548.024) 12.4439 ms (± 41.9621) 0.92
exec_bc_add 48.8289 ns (± 7.86601) 46.0393 ns (± 3.2155) 1.06
exec_bc_fibonacci 417.449 us (± 38.2494) 379.941 us (± 19.4119) 1.10
exec_bc_sum 92.8439 ms (± 1.84672) 93.9001 ms (± 3.77943) 0.99
exec_tbc_add 27.5315 ns (± 4.44154) 30.199 ns (± 1.88871) 0.91
exec_tbc_fibonacci 142.315 us (± 6.80183) 151.352 us (± 4.64195) 0.94
exec_tbc_sum 36.8369 ms (± 326.288) 40.0002 ms (± 72.8869) 0.92
exec_asmjit_add 3.58183 ns (± 0.568926) 3.54852 ns (± 0.303553) 1.01
exec_asmjit_fibonacci 14.4378 us (± 1.66988) 18.7673 us (± 1.48128) 0.77
exec_asmjit_sum 2.80428 ms (± 35.115) 3.50004 ms (± 267.36) 0.80
exec_bc_add_passesOff 49.4034 ns (± 7.90196) 45.8908 ns (± 3.50375) 1.08
exec_bc_add_passesOn 48.9036 ns (± 6.47111) 45.9572 ns (± 3.51532) 1.06
exec_bc_fibonacci_passesOff 407.745 us (± 12.3679) 385.434 us (± 29.5968) 1.06
exec_bc_fibonacci_passesOn 377.664 us (± 15.2924) 372.77 us (± 21.7772) 1.01
exec_bc_sum_passesOff 92.9661 ms (± 2.02795) 93.4191 ms (± 596.761) 1.00
exec_bc_sum_passesOn 79.3477 ms (± 1.21) 81.3862 ms (± 529.254) 0.97
exec_tbc_add_passesOff 27.5658 ns (± 4.09876) 30.0322 ns (± 1.39232) 0.92
exec_tbc_add_passesOn 27.6354 ns (± 4.75164) 30.0734 ns (± 1.47445) 0.92
exec_tbc_fibonacci_passesOff 141.813 us (± 6.52638) 150.86 us (± 4.2744) 0.94
exec_tbc_fibonacci_passesOn 143.575 us (± 8.36326) 150.878 us (± 4.27883) 0.95
exec_tbc_sum_passesOff 36.8387 ms (± 414.491) 40.1829 ms (± 1.08397) 0.92
exec_tbc_sum_passesOn 33.9061 ms (± 111.036) 37.8982 ms (± 190.298) 0.89
exec_asmjit_add_passesOff 3.54822 ns (± 0.396883) 3.55136 ns (± 0.243687) 1.00
exec_asmjit_add_passesOn 3.60573 ns (± 0.675701) 3.57812 ns (± 0.320122) 1.01
exec_asmjit_fibonacci_passesOff 14.6521 us (± 2.56231) 18.8925 us (± 1.61607) 0.78
exec_asmjit_fibonacci_passesOn 14.6181 us (± 2.50904) 18.8621 us (± 1.43634) 0.77
exec_asmjit_sum_passesOff 2.81619 ms (± 73.496) 3.4171 ms (± 181.353) 0.82
exec_asmjit_sum_passesOn 2.80463 ms (± 36.628) 3.36616 ms (± 49.79) 0.83
exec_bc_add_noRegAlloc 49.3046 ns (± 7.27714) 46.4357 ns (± 4.22175) 1.06
exec_bc_add_regAlloc 48.6708 ns (± 7.22537) 46.7642 ns (± 5.31525) 1.04
exec_bc_fibonacci_noRegAlloc 410.838 us (± 15.0025) 395.91 us (± 15.724) 1.04
exec_bc_fibonacci_regAlloc 410.875 us (± 25.2153) 376.486 us (± 8.17908) 1.09
exec_bc_sum_noRegAlloc 92.3896 ms (± 251.538) 91.6785 ms (± 1.39113) 1.01
exec_bc_sum_regAlloc 92.6358 ms (± 1.27845) 93.4326 ms (± 696.66) 0.99
exec_bc_add_call 51.0297 ns (± 11.8845) 45.6594 ns (± 1.78174) 1.12
exec_bc_add_switch 48.1783 ns (± 6.31992) 44.3743 ns (± 3.61594) 1.09
exec_bc_add_threaded 42.4174 ns (± 7.69045) 40.2668 ns (± 4.38769) 1.05
exec_bc_fibonacci_call 410.532 us (± 17.9191) 392.326 us (± 114.38) 1.05
exec_bc_fibonacci_switch 406.953 us (± 25.1002) 351.021 us (± 34.6609) 1.16
exec_bc_fibonacci_threaded 440.85 us (± 11.4232) 482.181 us (± 11.9623) 0.91
exec_bc_sum_call 92.5081 ms (± 683.314) 93.8768 ms (± 3.74557) 0.99
exec_bc_sum_switch 89.1564 ms (± 697.737) 73.8497 ms (± 901.46) 1.21
exec_bc_sum_threaded 88.1542 ms (± 1.49814) 96.3831 ms (± 2.83679) 0.91
exec_bc_add_threaded_noReuse 42.9816 ns (± 8.86021) 39.904 ns (± 4.54411) 1.08
exec_bc_add_threaded_reuse 34.1076 ns (± 7.13116) 32.7816 ns (± 3.48819) 1.04
exec_bc_fibonacci_threaded_noReuse 440.481 us (± 11.5459) 484.521 us (± 29.437) 0.91
exec_bc_fibonacci_threaded_reuse 441.483 us (± 13.3958) 486.007 us (± 39.2329) 0.91
exec_bc_sum_threaded_noReuse 88.2028 ms (± 1.63182) 96.3143 ms (± 2.62806) 0.92
exec_bc_sum_threaded_reuse 88.1011 ms (± 1.36325) 96.0535 ms (± 5.40093) 0.92
exec_bc_add_threaded_noSuperinstr 42.4751 ns (± 8.01464) 39.3846 ns (± 4.02588) 1.08
exec_bc_add_threaded_superinstr 43.0983 ns (± 10.0403) 39.2028 ns (± 3.39013) 1.10
exec_bc_addOne 41.9889 ns (± 8.85996) 38.584 ns (± 4.03121) 1.09
exec_mlir_addOne 300.328 ns (± 27.8986) 304.634 ns (± 8.23598) 0.99
exec_cpp_addOne 4.18026 ns (± 0.920204) 4.15724 ns (± 0.559822) 1.01
exec_interpreted_addOne 40.2853 ns (± 2.9755) 39.0111 ns (± 1.95301) 1.03
ir_add 836.72 ns (± 154.212) 726.721 ns (± 47.7483) 1.15
ir_ifThenElse 1.69401 us (± 167.171) 1.54533 us (± 102.518) 1.10
ir_deeplyNestedIfElse 3.52142 us (± 346.382) 3.30913 us (± 177.236) 1.06
ir_loop 1.79929 us (± 269.621) 1.58929 us (± 106.751) 1.13
ir_ifInsideLoop 3.00801 us (± 385.5) 2.80247 us (± 217.748) 1.07
ir_loopDirectCall 1.93511 us (± 317.841) 1.77846 us (± 143.778) 1.09
ir_pointerLoop 2.10336 us (± 202.846) 1.94516 us (± 123.311) 1.08
ir_staticLoop 1.53693 us (± 192.31) 1.43584 us (± 92.4984) 1.07
ir_fibonacci 1.88643 us (± 233.045) 1.69333 us (± 87.115) 1.11
ir_gcd 1.63595 us (± 232.859) 1.44132 us (± 78.6976) 1.14
ir_nestedIf10 8.0694 us (± 1.06178) 7.8232 us (± 441.241) 1.03
ir_nestedIf100 90.2702 us (± 6.23407) 94.1614 us (± 3.36158) 0.96
ir_chainedIf10 12.2607 us (± 1.66704) 11.9207 us (± 708.151) 1.03
ir_chainedIf100 168.764 us (± 9.6783) 173.224 us (± 6.80178) 0.97

This comment was automatically generated by workflow using github-action-benchmark.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@github-actions github-actions[bot] github-actions[bot] left review comments

Assignees

No one assigned

Labels

codex

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@PhilippGrulich