[CI test] pyalps: nanobind 3 split mode — one abi3 wheel per platform - #3
Open
skilledwolf wants to merge 67 commits into
Open
[CI test] pyalps: nanobind 3 split mode — one abi3 wheel per platform#3skilledwolf wants to merge 67 commits into
skilledwolf wants to merge 67 commits into
Conversation
The sse4 worker reads EPSILON (default 0) and floors it to 1e-6
regardless of the Hamiltonian. The per-bond diagonal constant is
c(type) = epsilon + max_diag_me(type), so the vertex carrying the
maximum diagonal matrix element is inserted with weight
c - me = epsilon ~= 1e-6.
On a sign-problematic (frustrated / non-bipartite) Hamiltonian this
makes the diagonal update effectively non-ergodic: the operator-string
sampler freezes, the expansion order <n> is undersampled, and the
simulation reports a confidently wrong, seed-dependent energy with
small error bars. A 3x3 Heisenberg lattice with periodic boundary
conditions gives <H> = -3.39 at beta=1 against the exact thermal
average -2.6525 (checked with two independent exact
diagonalizations). The constructor already warned about exactly this
case ("make sure that EPSILON is ergodic") but proceeded with the
non-ergodic floor.
The SSE energy estimator is invariant to EPSILON (a constant shift of
the diagonal vertex weights), so when EPSILON is unset and the model
is signed, default it to the largest diagonal matrix element. That
gives the maximum-diagonal vertex an O(1) insertion weight and
restores ergodicity, auto-scaling with the model. Sign-free models
keep the previous 1e-6 floor bit-identically (they are ergodic at
EPSILON=0, and a larger shift would only inflate the expansion
order), so existing unfrustrated results are unchanged. An explicit
EPSILON still overrides everything, as before. The warning is
reworded into a note describing the applied default.
Root-caused and fixed in the ALPS modernization fork while chasing
wrong SSE energies on periodic 2-D lattices; instrumentation first
refuted a bond-counting hypothesis and then identified the frozen
diagonal update. Verified there against exact diagonalization on a
periodic 3x3 Heisenberg lattice (SSE reproduces the exact <H> within
error bars after this change, with no EPSILON set) plus unchanged
results on chains and open 2-D/3-D lattices.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Relocate the CMake modules and build helpers from config/ to cmake/ (git-tracked renames, contents unchanged) and drop dead legacy: - remove Debian sid packaging (SVN/wheezy-era, unreferenced; superseded by the wheel + CPack packaging paths) - remove SVN-era license-header tooling (preamble*.in, update_preamble*) - remove stale root files (README.txt, README-package.txt, Welcome.txt, CTestConfig.cmake) - add CITATION.md and CMakePresets.json The config/ directory is now gone; cmake/ is the single home for build configuration. No functional/build behavior change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
version.h.in declared ALPS_VERSION_MAJOR/MINOR/PATCH with cmakedefine, which emits "/* #undef NAME */" when the substituted value is false-y. CMake counts 0 as false-y, so any x.y.0 release generated a header with that component silently missing. 2.4.0 would have tripped it. Use a plain #define for every macro the build unconditionally sets. Keep cmakedefine only for ALPS_XML_ALTERNATE_DIR, which the build never sets and parser/xslt_path.C guards with #ifdef. Also add ALPS_VERSION_NUMBER/ALPS_VERSION_NUM() for preprocessor version comparisons (BOOST_VERSION packing), and drop two macros: ALPS_SVN_REVISION, which expanded a variable unset since the SVN migration and was always #undef, and ALPS_SRCDIR, which baked the build machine's source path into an installed header for one line of pconfig output. Refs ALPSim#95 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The version was hardcoded in CMakeLists.txt and had already drifted: CMake said 2.3.3, the newest tag is v2.3.4, and pyalps' pyproject.toml says 2.3.4b1. Put the numeric release in ALPS_VERSION.txt, read it in cmake/ALPSVersion.cmake before project(), and derive ALPS_VERSION_MAJOR/MINOR/PATCH from PROJECT_VERSION_*. The file holds MAJOR.MINOR.PATCH and nothing else, because project(VERSION) rejects non-numeric input and neither SOVERSION nor find_package() matching has any notion of prerelease ordering. A malformed file is rejected with a message naming the file, rather than CMake's bare "VERSION format invalid". Corrects the version to 2.3.4 in passing. ALPS_VERSION_BUILD, which was always empty, becomes the ALPS_VERSION_PRERELEASE cache variable: it carries "beta.2" into display strings while the numeric version stays clean. "Prerelease" because a later change adds real build metadata (a git hash), and two similarly-named slots would confuse. Generate and install ALPSConfigVersion.cmake. Without it find_package(ALPS <version>) accepted any version it found and silently discarded the constraint. SameMinorVersion: within 2.3.x a patch release is drop-in, a minor bump is not guaranteed to be. Note this is stricter than the SOVERSION of MAJOR alone advertises; reconciling the soname is a packaging-visible change and is left alone here. Derive ALPS_YEAR with string(TIMESTAMP), which honours SOURCE_DATE_EPOCH, so distro and conda reproducible builds still get a stable year. ALPS_SRCDIR is dropped from the installed header by the preceding commit, but two tests use it to locate reference .h5 inputs. Give those two targets a private compile definition instead: a build-tree path belongs there, not in an installed public header. Refs ALPSim#95 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
CIBW_ENVIRONMENT does not expand the {project} placeholder (only
before-all/before-build/test/repair commands do), so ALPS_DIR and
CCACHE_DIR were set to the literal string "{project}/...". The wheel
build's find_package(ALPS REQUIRED CONFIG) then could not locate the
ALPSConfig.cmake installed by CIBW_BEFORE_ALL, failing CMake configure.
Use $(pwd), which cibuildwheel evaluates in the build environment
(cwd=/project in the Linux container, repo root on macOS) to the same
directory where _build/cibw-install lives. Validated end-to-end with a
local manylinux_2_28_aarch64 build: wheel builds, repairs, 14 tests pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
musllinux (Alpine/musl) lacks dnf, glibc's SunRPC/XDR, and execinfo. Branch before_all to apk; install libtirpc for ALPS's system-XDR path (ALPS_HAVE_RPC_XDR_H) and disable the execinfo backtrace. Validated end-to-end on musllinux_1_2_aarch64: wheel builds, auditwheel bundles libtirpc, 14/14 tests pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Reduce the MIT-license cleanup to the changes that need human review, deferring the ~220 mechanical header swaps to follow-up script-driven passes: - Delete the obsolete license-preamble templates and generator scripts (config/preamble*.in, config/update_preamble*). - Remove the legacy ALPS license check from alps_inspect (script/license_check.cpp and its registration in inspect.cpp). - Update user-visible license messages to MIT in alps::print_copyright/print_license, and delegate to alps::print_license where callers duplicated the old wording (parapack, looper). - Drop a stale "consult the web page for license details" line and reword the nonexistent "ALPS cite-me license" as a citation request. - Align README-package.txt, the Debian copyright file, the looper HTML docs, and the ja ED-03 notebook output with the repository MIT license. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
script/relicense.py mechanically replaces obsolete ALPS Library/Application license blocks and full MIT text in headers explicitly identified as ALPS Project headers. It emits a concise ALPS Project link and SPDX MIT notice while preserving copyright attribution, boxed comment edges, line endings, and unclassified or third-party notices. The generated sweep is kept in follow-up PR ALPSim#124 so it can be reviewed by rerunning the script. tool/license.py (the lattice-preview About dialog) embedded the full text of the ALPS LIBRARY LICENSE v1.1 and displayed it to users; replace it with the MIT license text matching LICENSE.txt. Its comment header is left for the mechanical sweep. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
copyright print notices for hybridization, interaction, framework
Align license messaging with MIT and add SPDX migration tooling (merge after ALPSim#123)
Replace ALPS license boilerplate with project and SPDX notices (merge after ALPSim#109)
qmc/sse: default EPSILON to an ergodic value for sign-problematic models
Conflicts: README-package.txt and config/debian/sid/copyright were reworded on master by the SPDX license sweep but are deleted by this branch; both deletions are kept. The citation guidance formerly in README-package.txt lives in CITATION.md on this branch. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Conflicts: tool/config.py.in, tool/license.py, and tool/preview.py were touched on master by the licensing sweep but are deleted by this branch; all three deletions are kept, as this branch removes the lattice-preview GUI as a unit. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…-wheel Conflict-free sync with the updated base branch, which now contains current master (SPDX license sweep and SSE epsilon fix). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The nanobind migration left the mcanalyze free functions unbound even though pyalps.alea still dispatches to them through alea_detail: autocorrelation_distance/_limit, cut_head_distance/_limit, cut_tail_distance/_limit, exponential_autocorrelation_time_distance/ _limit, uncorrelated_error, and binning_error all raised AttributeError at call time. Restore them with the same instantiation set as the Boost.Python module (scalar and vector mcdata/mctimeseries/ mctimeseries_view where each was previously exposed). The exponential fit helpers return StdPairDouble so the documented fit.first / fit.second attribute API keeps working, and integrated_autocorrelation_time now accepts that StdPairDouble as well as a plain 2-tuple, as its comment already promised. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The standalone wheel dropped the XML/XSL data path: pyalps_config.py was checked in with hardcoded-empty install dirs and its template was never configured, and the wheel shipped no pyalps/xml directory, so pyalps.tools stylesheet and lattice/model-library workflows regressed against the legacy wheel build. Install the stylesheets plus the lattice and model libraries into pyalps/xml as the legacy ALPS_PYTHON_WHEEL build did, generate pyalps_config.py at build time with fallback paths pointing at the ALPS SDK the build used, drop the stale checked-in copy, and vendor lib/xml into the sdist so wheels rebuilt from it bundle the same files. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
v4 targets Node.js 20, which GitHub runners now warn is deprecated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
3.21 was chosen as the CMakePresets v3 schema floor, but no supported distro ships exactly 3.21, so that floor is never exercised. 3.22 is what Ubuntu 22.04 (the oldest CI platform) ships, making the declared minimum one that CI actually builds with. RHEL 9 (3.31) and Debian 12 (3.25) are unaffected. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
C++17 added std::size and std::data. Arguments whose types carry std template arguments (std::vector, mcdata<std::vector<T>>, ...) pull namespace std in via ADL, making unqualified size()/data() calls that previously resolved to alps::size/alps::data ambiguous. Qualify the call sites in alea/mcanalyze.hpp, alea/mcdata.hpp, numeric/vector_valarray_conversion.hpp, and test/alea/mcanalyze.C. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
non_pod's mixed-type friends delegated T==non_pod to non_pod==T, which under C++20 operator rewriting resolves back to the same friend as a reversed candidate: infinite recursion (a hang at -O3, stack overflow at -O0). test_deque/test_vector timed out at 600 s in the first real C++23 CI run. Compare data_ directly instead. Verified with g++ 16 -std=c++23 -O3: both tests pass. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
CI has been sweeping C++11-23 by passing -std=c++XX via CMAKE_CXX_FLAGS, but the unconditional set(CMAKE_CXX_STANDARD 14) appended its own -std flag afterwards, so every job silently built C++14. CONTRIBUTING.md has claimed C++17 all along; this makes the build match the docs. - CMakeLists.txt now defaults CMAKE_CXX_STANDARD to 17 and honors -DCMAKE_CXX_STANDARD=20/23; CI passes the standard that way instead of through CMAKE_CXX_FLAGS. The sweep tests 20/23 on top of the 17 baseline (the 11/14 entries are gone with the floor raise). - Remove ' throw (std::runtime_error)' exception specifications in src/ietl/krylov_wrapper.h (ill-formed since C++17) and the dead, never-included src/boost/function_objects.hpp (std::binary_function was removed in C++17). - Drop GCC 10 / Clang 13 from the matrix; GCC 11 / Clang 14 (Ubuntu 22.04 / RHEL 9 defaults) are the new tested floor. - Delete requirements.txt: unreferenced, and its numpy<2.1 ceiling contradicted pyproject.toml and CI reality. Verified on the skilledwolf/ALPS fork: full builds green at C++17/20/23 with GCC 11-16 and Clang 14-22. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
# Conflicts: # README-py.md
Restore legacy HDF5 list save semantics (audit issue 13): exact-type homogeneous rectangular list trees are written as one N-D dataset that keeps the element type ([1,2,3] stays int32, floats stay float64, out-of-int32-range widens to int64), equal-shape numpy-array lists stack via numpy, and bool-containing / mixed-type / ragged lists fall back to the legacy per-index group descent. The previous probe ladder cast with implicit conversion enabled and double first, so integer lists were silently written as float64. Unify the three dict->params converters into dict_to_params.hpp (issue 17): params, mcbase and the application modules now ingest values identically; oversized ints raise instead of silently truncating through paramvalue's 32-bit int; int lists round-trip as ints; complex scalars are supported; None is rejected with a message that names it (issue 8). Copy __eq__/__ne__/__hash__ onto the MutableMapping shims (issue 1): the hasattr guard could never copy them (object provides both), so mapping equality was lost relative to the Boost.Python __bases__ inheritance. Move observable.__lshift__ from a type monkeypatch into the C++ binding, returning self for chaining (issue 2). Forward save/load through the mcbase trampoline (issue 9) so Python overrides are reached by C++ virtual dispatch; slot count pinned to the five virtuals in src/alps/mcbase.hpp. Smaller items: cache the numpy module and use limited-API PyTuple_SetItem in numpy_compat.hpp (issues 24, 29); release the previous entry on archive-exception re-registration (issue 12); in-place accumulator result operators use rv_policy::none + is_operator (issue 11); document copy semantics on the dwa worldlines accessors (issue 7) and overload ordering in pyalea (issue 14); mirror libalps' BOOST_* config defines in the bindings build (issue 16); replace the dead pyalps.mpi import chain with a clear ImportError (issue 18); cap nanobind below the next major (issue 19). Issues 10/43 (duplicate wrapper) and the issue-11 leak were checked empirically against nanobind 2.15 and refuted; comments record the verified behaviour. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
pyhdf5io_test and mcdata_test consisted of print() calls diffed by a CTest harness that no longer exists, so the wheel CI's pytest run passed vacuously — exactly why the HDF5 int->float64 regression went unnoticed. Rewrite both with assertions derived from the historic .output fixtures (which are removed along with the dead run_python_test.cmake), including dtype checks for every list shape the old build distinguished, plus regression cases for bool/mixed/ int64-range lists. Extend test_binding_surface with regression tests for mapping equality and the params value ladder, observable << chaining, save/load overrides reached through C++ virtual dispatch, and in-place accumulator result identity. mcdata's unary minus expectations document a long-standing libalps bug (mcdata<T>::operator-() returns *this unchanged) that the old fixture also recorded; flip them when the C++ operator is fixed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The nanobind bindings in bindings/python/pyalps fully replace the
Boost.Python modules, but the old sources stayed in the tree: nothing
built them, they declared the same module and type names as the
shipped extensions, and they could no longer compile anyway because
ALPS_HAVE_PYTHON (and paramvalue's boost::python::object variant
alternative) is not emitted by any build. Remove them (audit issue 3):
- src/alps/python/ and src/alps/ngs/python/ (the module sources)
- src/alps/hdf5/python.{hpp,cpp}, src/alps/ngs/boost_python.hpp,
src/alps/ngs/detail/{export_sim_to_python,get_numpy_type,
extract_from_pyobject}.hpp, src/alps/ngs/lib/get_numpy_type.cpp
- src/boost/mpi/module.cpp (the never-built mpi_c source)
- applications/qmc/dwa/python/dwa.cpp (superseded by
bindings/python/pyalps/cpp/apps/dwa.cpp)
- the unbuilt Boost.Python export tutorials
(tutorials/ngs/5_export_python, the code-07 export.{cpp,py} files)
Collapse the now-unreachable ALPS_HAVE_PYTHON conditionals in the
surviving headers and sources (mcanalyze, mcdata, value_with_error,
params, paramvalue, paramvalue_reader, scheduler/proto/mcbase), which
removes the boost::python::object declarations for good. No reference
to boost::python remains outside explanatory comments.
Validated: full wheel-deps SDK rebuild, pyalps wheel rebuild against
it, and 22/22 Python tests green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Make the two ABI non-decisions deliberate and visible (audit issues 21-23, 30-31, 42): the extension modules intentionally do not declare free-threading support — importing pyalps on 3.13t/3.14t re-enables the GIL, which is required while libalps uses the GIL as its lock around shared state (mcobservable's refcount table, the ngs::signal singleton, mcdata's lazy statistics) — and per-version wheels are kept instead of abi3, though the bindings are kept free of limited-API violations so stable-ABI builds remain an option. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A ten-angle review plus empirical edge testing of the audit-response
commits surfaced real gaps, all fixed and regression-tested:
- Group-saved lists with 11+ elements loaded back as dicts (or out of
order): H5Literate yields child names lexicographically, but the
loader compared them positionally against '0','1',... Recover list
shape from the name SET {0..n-1} and load in numeric order, as the
legacy loader effectively did by indexing value[cast<size_t>(name)].
- Lists of numpy scalars (np.int64, np.float32, np.bool_, ...) lost
their legacy vectorization into one typed dataset, and rectangular
ndarray/sequence mixes no longer stacked. numpy_stackable() now
delegates both shapes through numpy.asarray — vetoing trees that
contain plain bool leaves, which numpy would silently promote to
0/1 (the legacy rules always grouped those).
- Re-saving a group-shaped list or dict over an existing group kept
stale children (create_group is a no-op on an existing group);
legacy wiped the group first. Both branches now delete it.
- The params ladder silently widened out-of-int32 integers inside
lists to double (corrupting values beyond 2^53) while raising for
scalars, missed numpy bool scalars in its bool guard (stored as
1.0/0.0), and rejected numpy integer scalars while accepting numpy
floats. One pre-scan now range-checks every integral element
(PyNumber_Index covers numpy ints), numpy bools count as bools, and
numpy integer scalars are accepted consistently as scalars and in
lists.
- observables lacked __delitem__, so the copied MutableMapping
pop/popitem/clear raised TypeError; the legacy map_indexing_suite
provided deletion. Bound it (mcobservables derives from std::map).
- numpy_module()'s magic static could deadlock two GIL-juggling
first callers; replaced with an atomic double-check. NULL results
of PyTuple_New/PyLong_FromUnsignedLongLong are now checked.
- The bindings' Boost config defines are now lifted from the
SDK-exported ALPS_CMAKE_CXX_FLAGS instead of hand-mirrored.
- Smaller items: params_getitem regained its defined() fast path for
misses, list_vectorizer dropped derivable state and uses exact
numeric type checks (numpy scalar handling moved to the stacking
path), orphaned includes removed from mcbase.cpp, guard-collapse
blank-line scars squeezed, mcdata assertions tightened to
assert_allclose.
Validated: wheel rebuild against the SDK, 23/23 Python tests.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Consolidate build configuration under cmake/
Require C++17 and fix the CI standard matrix (after ALPSim#112)
Bump nanobind to >=3.0,<4 and switch every extension module from NB_STATIC to split mode (BACKEND_MODULE nanobind_backend). The modules now compile under Py_LIMITED_API (3.10 floor), carry only the nanobind frontend, and resolve the compiled backend at import time from the new nanobind-backend runtime dependency; the wheel is tagged cp310-abi3 and one wheel per platform covers CPython 3.10+. Limited-API adaptations: - tp_name dispatch (PyTypeObject is opaque) goes through the new alps::python::type_fullname() helper, which reconstructs the tp_name spellings from __module__/__qualname__. - PyComplex_AsCComplex/Py_complex replaced with nanobind's backend-served complex caster (params) and the component getters (hdf5 list save). - NB_TRAMPOLINE size argument dropped (deprecated in nanobind 3). The downstream export tutorial builds in split mode too: extensions only share bound types (mcbase identity) when they share a backend. Packaging/CI: cibuildwheel builds cp310 only; the downstream-export test moves into that single build; the smoke_test job now installs the one wheel on CPython 3.10-3.14 per platform and runs the full suite; wheel consumers installing with --no-deps must add nanobind-backend. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
skilledwolf
force-pushed
the
modernization/pyalps-nanobind3-split
branch
from
August 26, 2026 05:51
39c1850 to
0cb345d
Compare
The fork's Actions policy (and supply-chain hardening generally) requires action references pinned to full-length commit SHAs; the version each SHA resolves to is noted inline. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
skilledwolf
force-pushed
the
modernization/pyalps-nanobind3-split
branch
from
August 26, 2026 05:54
0cb345d to
9de7731
Compare
…ists The split-mode wheel depends on nanobind-backend at import time, and that package publishes neither musllinux wheels nor an sdist, so the repaired musllinux wheel cannot install. Skip musllinux until the backend covers musl; documented in the README. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fork-internal CI run for the nanobind 3.0 upgrade + split mode on top of ALPSim#107 (
modernization/pyalps-nanobind-wheel). Not for merging here — validation only.BACKEND_MODULE nanobind_backend), compiled underPy_LIMITED_API(3.10 floor)cp310-abi3wheel per platform;nanobind-backendbecomes a runtime dependencytp_namedispatch viaalps::python::type_fullname(),PyComplex_AsCComplexremoved, trampoline macro updatedLocal validation (macOS arm64): full suite passes on 3.11–3.14 incl. downstream export; on 3.10 the HDF5/params/mcdata suites pass (full-suite 3.10 blocked locally only by scipy's cp310 wheel failing to dlopen on Darwin 27 — covered by this CI).
🤖 Generated with Claude Code