Skip to content

Split extxyz into ASE-free core + ase-extxyz plugin (v0.3.0)#28

Merged
jameskermode merged 2 commits into
masterfrom
refactor/ase-plugin-split
Apr 29, 2026
Merged

Split extxyz into ASE-free core + ase-extxyz plugin (v0.3.0)#28
jameskermode merged 2 commits into
masterfrom
refactor/ase-plugin-split

Conversation

@jameskermode
Copy link
Copy Markdown
Member

Breaking change for v0.3.0 — drops the ASE dependency from extxyz and adds a sibling ase-extxyz plugin package that wires the C parser into ase.io via the external I/O plugin entry point.

What

Package Lives Depends on Public API
extxyz python/extxyz/ numpy, pyleri Frame, iread_dicts, read_dicts, write_dicts, cextxyz (ctypes)
ase-extxyz python/ase-extxyz/ ase>=3.23, extxyz>=0.3.0 None for direct use; registers cextxyz format with ase.io

After this lands users do:

# C parser only (no ASE)
import extxyz
for frame in extxyz.iread_dicts('traj.xyz'):
    ...

# Via ASE
import ase.io
atoms = ase.io.read('traj.xyz', format='cextxyz')
ase.io.write('out.xyz', atoms, format='cextxyz')

The pre-0.3 top-level names (extxyz.read, extxyz.write, extxyz.iread, extxyz.ExtXYZTrajectoryWriter) raise ImportError with a message pointing at ase-extxyz. ExtXYZTrajectoryWriter is kept on the new package, now keeping a single libc FILE* open across .write() calls (the v0.2.x version held a Python file handle and silently fell back to pure-Python — strictly slower than this).

How

  • python/extxyz/extxyz.py (~870 lines, mixed) split into grammar.py (parser AST + Properties + encoders) and core.py (Frame + iread_dicts/write_dicts).
  • python/extxyz/utils.py deleted; SinglePointCalculator glue + Atoms ↔ Frame translation moved to python/ase-extxyz/ase_extxyz/io.py.
  • python/extxyz/cli.py rewritten to emit JSON via ExtXYZEncoder so the extxyz console script works in a no-ASE install.
  • All ASE-using tests (tests/test_kv_parsing_*, test_ase_cases, test_traj_writer) git mv-d to python/ase-extxyz/tests/ and updated to import from ase_extxyz.io / conftest.read. The original tests/conftest.py followed.
  • New tests/test_core.py covers the dict API plus a subprocess check that import extxyz does not pull in any ase.* module.

Plugin contract details

Format name: cextxyz (can't shadow ASE's built-in extxyz/xyz).
IOFormat code: +S — multi-frame, string filename. The C parser opens via libc; if ASE pre-opened a Python handle (+F) the C path would be unreachable.
ext intentionally not set — users opt in with format='cextxyz', no auto-detection clash with the built-in.

CI

  • New .github/workflows/ase-extxyz.yml: matrix tests + sdist/wheel publish on ase-extxyz-v* tags.
  • python-package.yml updated to install both packages and run both pytest suites (with USE_FORTRAN=T for the fextxyz round-trip).
  • build-wheels.yml unchanged; its tags: ['v*'] glob doesn't match ase-extxyz-v*, so the two release pipelines stay independent.

Verified locally (macOS, Python 3.13, ASE 3.28)

  • ASE-free venv: 10/10 tests/ pass, pip list shows no ase after pip install ./.
  • Both-installed venv: 48 pass, 2 skipped (pre-existing 2D string-array cases). assert 'cextxyz' in ase.io.formats.ioformats succeeds. Round-trip via ase.io.write/read matches positions to 1e-7.

Release sequencing (after merge — not part of this PR)

  1. Tag v0.3.0 on master → build-wheels.yml publishes extxyz==0.3.0 to PyPI.
  2. Wait ~1 min for PyPI to index it.
  3. Tag ase-extxyz-v0.1.0 on the same SHA → ase-extxyz.yml publishes ase-extxyz==0.1.0, depending on extxyz>=0.3.0.

Both tags can be cut from the same commit; they just need to land in order so pip install ase-extxyz can resolve the extxyz dep.

Test plan

  • CI Python 3.10–3.13 build green on python-package.yml (both extxyz + ase-extxyz tests, plus Fortran round-trip)
  • CI Python 3.10–3.13 build green on ase-extxyz.yml
  • build-wheels.yml still green for the extxyz package (16 wheels)

🤖 Generated with Claude Code

jameskermode and others added 2 commits April 29, 2026 18:14
Drops the ASE dependency from extxyz and adds a sibling ase-extxyz
package that wires the C parser into ase.io's plugin machinery as the
'cextxyz' format. Rationale & full plan: see the approved design doc.

extxyz (this package) — breaking changes for v0.3.0
- Drops ase>=3.17 from project.dependencies. extxyz now depends only
  on numpy + pyleri.
- Removes the ASE-aware top-level API: extxyz.read, write, iread, and
  ExtXYZTrajectoryWriter. Calling any of those now raises ImportError
  with a message pointing at ase-extxyz.
- New dict/array-based public API (ASE-free):
    extxyz.Frame                  — dataclass: natoms, cell, pbc, info, arrays
    extxyz.iread_dicts(...)       — generator
    extxyz.read_dicts(...)        — eager
    extxyz.write_dicts(...)       — accepts Frame or iterable of Frame
- python/extxyz/extxyz.py is now a tiny migration shim. The parser AST
  classes, Properties, and the comment-line encoders moved to
  python/extxyz/grammar.py. The frame-level read/write moved to
  python/extxyz/core.py.
- python/extxyz/utils.py deleted (its contents — SinglePointCalculator
  and Calculator-results glue — moved into ase-extxyz).
- python/extxyz/cli.py rewritten to dump JSON via ExtXYZEncoder; no
  ASE in its import closure.

ase-extxyz (new sibling package, python/ase-extxyz/)
- pyproject.toml declares the ASE.ioformats entry point pointing at
  ase_extxyz.io:cextxyz_format. Code "+S" (multi-frame, string path),
  no ext set — users opt in with format='cextxyz' to avoid clashing
  with ASE's built-in 'extxyz'.
- ase_extxyz.io exposes:
    cextxyz_format       — ExternalIOFormat instance
    read_cextxyz(...)    — generator yielding Atoms; honours ASE's
                           index= argument including index=-1 default
    write_cextxyz(...)   — writes one or many Atoms
    ExtXYZTrajectoryWriter — keeps a single libc FILE* open across
                             writes, suitable for opt.attach(traj)
- Translation Frame ↔ Atoms lifted from extxyz: per-atom name remap
  (pos↔positions, species↔symbols, velo↔momenta with mass-aware
  unit conversion), Properties.from_atoms equivalent, optional
  SinglePointCalculator from comment-line keys.

Tests
- All ASE-using tests (kv_parsing_*, ase_cases, traj_writer) moved to
  python/ase-extxyz/tests/ and updated to import from
  ase_extxyz.io / conftest.read instead of the removed extxyz.read.
- New tests/test_core.py covers the dict API, plus a subprocess check
  that `import extxyz` does not pull in any ase.* module.

CI
- New .github/workflows/ase-extxyz.yml: matrix tests on ubuntu-latest
  for 3.10..3.13 (installs both packages from the source tree, runs
  pytest); publish job on tag ase-extxyz-v* builds an sdist + wheel
  via flit_core, attaches to the GitHub Release, and uploads to PyPI.
- python-package.yml updated to install both packages, then run the
  extxyz core tests and the ase-extxyz tests (with USE_FORTRAN=T for
  the fextxyz round-trip).
- build-wheels.yml unchanged — its `tags: ['v*']` glob doesn't match
  ase-extxyz-v* (different prefix), so the two release pipelines stay
  independent.

Local verification (matches the plan's checklist)
- extxyz alone in a fresh venv with NO ase installed: 10/10 core tests
  pass, `pip list` shows no ase.
- extxyz + ase-extxyz + ase: ase.io.read('...', format='cextxyz') and
  ase.io.write round-trip; plugin registered (assert 'cextxyz' in
  ase.io.formats.ioformats); 48 tests pass, 2 skipped (pre-existing
  2-D string-array unsupported cases).

Tagging once merged: v0.3.0 publishes extxyz, then ase-extxyz-v0.1.0
publishes ase-extxyz (which depends on extxyz>=0.3.0 — the order
matters because PyPI must have extxyz 0.3.0 before the second tag's
build resolves the dep).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
In-tree extxyz reports a 0.2.2.devN+gSHA version (from
discover_version.py + git describe) until v0.3.0 is tagged.
ase-extxyz declares extxyz>=0.3.0 which 0.2.2.devN does not
satisfy by PEP 440 ordering, so pip resolves the dependency
from PyPI and finds only 0.2.x — fail.

Use --no-deps for the ase-extxyz install in both workflows
(python-package.yml and ase-extxyz.yml). The already-installed
in-tree extxyz is what ase_extxyz.io imports at runtime, so the
constraint is effectively satisfied; we just don't want pip to
re-resolve it.

Once v0.3.0 is tagged on master, discover_version reports 0.3.0
and the constraint resolves cleanly — at that point --no-deps
isn't strictly needed, but it doesn't hurt to keep.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@jameskermode jameskermode merged commit 3feb058 into master Apr 29, 2026
26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant