Skip to content

Python 3.14 free-threaded (cp314t) support; release 0.3.0#69

Merged
yokofly merged 4 commits into
developfrom
yokofly/cp314t-cython-3.2
Jun 12, 2026
Merged

Python 3.14 free-threaded (cp314t) support; release 0.3.0#69
yokofly merged 4 commits into
developfrom
yokofly/cp314t-cython-3.2

Conversation

@yokofly

@yokofly yokofly commented Apr 14, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Summary

  • All four Cython extensions declare freethreading_compatible = True; the GIL stays disabled on 3.14t (C sources regenerated with Cython 3.2.4)
  • FT smoke now asserts sys._is_gil_enabled() is False on free-threaded builds (runs on every built wheel via the cibuildwheel test-command)
  • New threaded stress tests: extension-level (no server, wired into the smoke-ft job) and parallel-clients (integration, runs in the dockerized matrix)
  • Version 0.3.0, CHANGELOG + README updated
  • Packaging: same proton-driver package name; cp314t wheels ship alongside cp314 (community convention — pip picks the right wheel per interpreter). Verified cibuildwheel 3.4.1 emits all cp314/cp314t manylinux+musllinux identifiers for x86_64 and aarch64.
  • Test-infra: version gating tolerates prerelease server versions (3.3.1-rc.13); parallel-inserts count uses plain SELECT since table() is not applicable to Memory-engine streams on newer servers

Free-threading audit

No module-level mutable state (largeint's MAX_UINT64/MAX_INT64 are immutable ints assigned once at init), no static C buffers (varint uses per-call stack arrays; reader/writer buffers are per-instance PyMem allocations), no cross-call borrowed-reference caching. Per-object thread-safety follows the standard "callers don't share one object across threads unsynchronized" contract.

Test plan

  • CI matrix green (including smoke-ft on 3.14t)
  • Full pytest suite under CPython 3.14.4t against a local proton-enterprise server (3.3.1-rc.13): zero failures attributable to the FT build — failure set is byte-identical to released 0.2.13 on CPython 3.12 against the same server (pre-existing server-version drift)
  • GIL verified disabled end-to-end: import all extensions + run real queries, sys._is_gil_enabled() stays False
  • 8-thread parallel queries/inserts pass under 3.14t against the live server
  • cibuildwheel 3.4.1 --print-build-identifiers includes cp314t-*

🤖 Generated with Claude Code

@yokofly yokofly changed the title Regenerate Cython outputs with 3.2.4 for CPython 3.14 support Python 3.14 free-threaded (cp314t) support; release 0.3.0 Jun 12, 2026
@yokofly yokofly marked this pull request as ready for review June 12, 2026 06:58
@coveralls

coveralls commented Jun 12, 2026 •
edited
Loading

Copy link
Copy Markdown

Coverage Report for CI Build 27418729863

Warning

No base build found for commit 70a1c63 on develop.
Coverage changes can't be calculated without a base build.
If a base build is processing, this comment will update automatically when it completes.

Coverage: 94.387%

Details

  • Patch coverage: 1 of 1 lines across 1 file are fully covered (100%).

Uncovered Changes

No uncovered changes found.

Coverage Regressions

Requires a base build to compare against. How to fix this →

Coverage Stats

Coverage Status
Relevant Lines: 3884
Covered Lines: 3666
Line Coverage: 94.39%
Coverage Strength: 12.44 hits per line
💛 - Coveralls

@yokofly yokofly force-pushed the yokofly/cp314t-cython-3.2 branch from 8837e9e to de0abe0 Compare June 12, 2026 10:22
yokofly and others added 2 commits June 12, 2026 03:54
@yokofly @claude
Add free-threaded CPython 3.14 support to all compiled extensions
6a562b4 
Regenerate the four Cython extensions (varint, largeint, bufferedreader,
bufferedwriter) with Cython 3.2.4 and declare freethreading_compatible in
each .pyx, so the GIL stays disabled when the driver is imported on the
free-threaded 3.14t interpreter. Build with CYTHON_COMPRESS_STRINGS=0 to
keep the extensions importable when stdlib compression modules (zlib/bz2)
are unavailable. setup.py refuses to regenerate with Cython < 3.1, which
silently ignores the directive and would ship GIL-re-enabling extensions.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@yokofly @claude
Add free-threading tests and adapt the suite to server 3.x
d081153 
New coverage: a block-import smoke script that asserts every extension
imports without stdlib compression modules and that the GIL stays
disabled on 3.14t, a threaded stress test hammering the compiled
extensions from 8 concurrent threads, and a parallel-clients
integration test (concurrent queries and inserts).

Suite fixes for drift accumulated since the tests were written:
tolerate prerelease server versions (3.3.1-rc.13), drop the shared test
stream defensively in tearDown, version-gate tests for behaviors that
changed in server 3.x (native JSON type, use_client_time_zone) via a
new skip_on_server_version_from helper, pin pandas<3 for the numpy
lanes, and skip mock-internals connect tests on pypy where cpyext
cannot see MagicMock attributes from Cython 3.2 code.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@yokofly yokofly force-pushed the yokofly/cp314t-cython-3.2 branch 2 times, most recently from f0ec819 to 3fc56a9 Compare June 12, 2026 11:04
@yokofly

yokofly commented Jun 12, 2026

Copy link
Copy Markdown
Collaborator Author

Review-response notes: Cython version guard

An external review flagged that rebuilding the extensions from .pyx with an older Cython would silently drop the free-threading declaration. Verified and fixed:

The hazard is real. Cython 3.0.12 compiles our directive-bearing .pyx files with exit 0 and no warning, and its output contains zero Py_mod_gil slots — importing such an extension on 3.14t re-enables the GIL process-wide. The committed .c files (generated with Cython 3.2.4) carry the slot for all four extensions.

Exposure is narrow. Standard pip install uses PEP 517 build isolation with no Cython in the build env, so setup.py compiles the committed .c as-is — FT preserved. Only direct setup.py runs or --no-build-isolation with an old Cython on the path would regenerate.

Fix: setup.py now refuses to cythonize with Cython < 3.1:

RuntimeError: Building from .pyx sources requires Cython >= 3.1 (found 3.0.12): older releases silently ignore the freethreading_compatible directive.

Verified both directions: rejects 3.0.12, builds normally with 3.2.4, and the no-Cython wheel path is untouched.

Two related accuracy fixes from the same review pass: the README caveat now correctly scopes the clickhouse-cityhash gap to cp314t (upstream 1.0.2.5 does ship regular cp314 wheels — the 3.14 CI lanes install them), and the changelog states explicitly that PyPy wheels are now pp311-only (pp38–pp310 are EOL; sdist still installs there).

🤖 Generated with Claude Code

@yokofly yokofly force-pushed the yokofly/cp314t-cython-3.2 branch 2 times, most recently from 28661db to 11a655c Compare June 12, 2026 13:24
yokofly and others added 2 commits June 12, 2026 06:28
@yokofly @claude
Repair CI and build Python 3.14 / 3.14t wheels in releases
0d45683 
The test workflow had not truly run since the ubuntu-20.04 runner
retirement. Move to ubuntu-22.04, replace setup.py develop with
pip-managed installs (easy_install ignored Requires-Python), unpin
coveralls, add a 3.14t smoke job (build + block-import + GIL-off
threaded stress), and add 3.13/3.14 to the matrix.

Server bring-up fixes: pin the FT-enabled timeplusd image, drop the
root-owned bind mount the non-root container cannot write to, probe
readiness through the right host with diagnostics on timeout, and cap
nativelog segment preallocation via a config.d drop-in (the default
preallocates 2GB per stream and filled the runner disk).

The release workflow gains cp314/cp314t wheel builds via cibuildwheel
and publishes an sdist again — no release since 0.2.10 shipped one, so
interpreters without a prebuilt wheel (e.g. EOL PyPy) had no
installable artifact. The GitHub release is created with the built-in
token (the GH_ACCESS_TOKEN PAT no longer exists), and wheels are
stored as artifacts before that step so a release failure cannot
strand the PyPI publish.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@yokofly @claude
Release 0.3.0: document free-threaded support
0423cae 
Bump version to 0.3.0 with a changelog entry. README documents the
free-threading story: the driver itself keeps the GIL disabled on
3.14t; the optional compression extras do not yet (lz4 4.4.5+ is
FT-ready, clickhouse-cityhash lacks cp314t wheels and is replaced by
the timeplus-io fork in a follow-up, zstd lacks the FT declaration
upstream).

cibuildwheel config: keep building the PyPy wheels modern toolchains
support (pp311; pp38-pp310 are EOL and fall back to the sdist) and
smoke-test every built wheel with the block-import script.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@yokofly yokofly force-pushed the yokofly/cp314t-cython-3.2 branch from 11a655c to 0423cae Compare June 12, 2026 13:28
@yokofly yokofly merged commit 4a37a1e into develop Jun 12, 2026
21 checks passed
@yokofly yokofly mentioned this pull request Jun 12, 2026
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@chenziliang chenziliang chenziliang approved these changes

@gangtao gangtao Awaiting requested review from gangtao

@yuzifeng1984 yuzifeng1984 Awaiting requested review from yuzifeng1984

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@yokofly @coveralls @chenziliang