python-bug-fixes

[ZachPipes]

Summary

Gets the project building again, replaces the mocked Python test suite with real
numerical tests, and fixes the packaging metadata needed to publish. In the
process the new tests uncovered 7 genuine defects in the C++ and Python
layers, which are documented as strict-xfail tests rather than silently fixed.

12 commits, 32 files, +3265 / −1313.

Why

main/develop could not be built at all. Three defects stacked:

• CMakeLists.txt required find_package(Python 3.14 REQUIRED)
• pybind11 was pinned via FetchContent to v2.12.1, which predates Python 3.13
  and 3.14 support — so the two requirements were mutually unsatisfiable on any
  machine
• setup.py passed -DPYTHON_EXECUTABLE, which modern CMake FindPython
  ignores, so CMake could bind to a different interpreter than the wheel was
  tagged for

Because nothing built, 11 of 13 Python test files had drifted to mocking the
_fastdist extension entirely — asserting that a MagicMock returned the value
it was handed. That made an entire class of bug undetectable.

Changes

Build

• Lowered the CMake Python floor to 3.12, matching what the README and
  build_all.ps1 already claimed to support
• Replaced the FetchContent pin with find_package(pybind11 CONFIG REQUIRED),
  resolved from the active environment's pip package, with a fallback to a
  system install (e.g. Debian pybind11-dev)
• Fixed setup.py to pass Python_EXECUTABLE (capital P) so CMake binds to
  sys.executable
• Development → Development.Module, the correct component for extension
  modules
• Removed the unused python/pybind11 submodule. It was never wired into the
  build — there was no add_subdirectory and no reference to the path anywhere.
  Reclaims 4.3 MB of working tree and 15 MB under .git/modules

Versioning

• project(fastdist VERSION x.y.z) in CMakeLists.txt is now the single source
  of truth. version.h is generated from a new version.h.in template via
  configure_file; bindings.cpp and setup.py both derive from it
• Previously the version was hand-written in five places and three of them
  disagreed (0.0.1 vs 0.1.0)
• Added fastdist.__version__ and test_version.py, which fails if the C++
  constant and the wheel metadata ever drift apart

Packaging

• Added install_requires=["numpy>=1.21"]. Previously pip install succeeded
  and import fastdist then failed with ModuleNotFoundError: numpy
• nvidia-ml-py moved to an optional [gpu] extra
• python_requires corrected from >=3.7 to >=3.12 (six modules use PEP 604
  syntax requiring 3.10+, and CMake requires 3.12)
• Real description, long_description from README, url, Apache-2.0 license
  metadata, and classifiers replacing the previous placeholders

Correctness

• Bernoulli._mgf_scalar / ._cgf_scalar called _core.bernoulli_mgf /
  _core.bernoulli_cgf, which are not bound. Both raised AttributeError on
  any call with valid arguments
• ChiSquare was importable from fastdist.distributions but missing from the
  top-level package exports
• validate_gpu_capacity raised NVMLError_Uninitialized out of its finally
  block on any machine without an NVIDIA driver. NVML now initializes once per
  process with an atexit teardown, degrades cleanly when unavailable, and
  accepts a device_index
• config.py no longer touches the filesystem at import. It previously created
  a directory and wrote a default config file on import fastdist, which fails
  outright in read-only containers and CI runners. The file is now written only
  by an explicit auto_tune() call

CI

• Workflows now run on PRs targeting develop, not just master
• Build job installs pybind11 via pip and pins
  -DPython_EXECUTABLE="$(which python)"
• Dropped submodules: recursive from both checkout steps
• __pycache__ and .pytest_cache added to .gitignore

Tests

The Python suite was rewritten to exercise the real compiled extension. Every
numeric assertion is made against an independently derived closed form or a
reference implementation, never a value captured from a previous run.

485 tests (mostly mocked) → 1745 tests (1686 passing, 59 documenting known
defects). Zero mocks remain.

New conftest.py provides shared tolerances, a known_bug marker, and
reference implementations of the regularized lower incomplete gamma and
regularized incomplete beta — validated against hand-computed values before use.

Beyond pointwise checks, each distribution gets structural property tests: PMFs
sum to 1, CDFs are monotonic/bounded/saturating, K(t) == ln M(t), M(0) == 1,
classmethods agree with instance methods, means match independently computed
PMF-weighted sums. Plus identities like exponential memorylessness,
Beta(1,1) ≡ uniform, Gamma(1,θ) ≡ exponential, and Γ(x+1) = xΓ(x).

Array-capable classes additionally cover dtype/shape, numpy input, empty arrays,
2-D and non-numeric rejection, and step_size semantics.

Defects found — NOT fixed in this PR

All are marked @pytest.mark.xfail(strict=True) with known_bug. The suite
stays green, and each becomes a hard failure the moment it's fixed and its
marker goes stale, so none can be fixed and forgotten.

#	Defect	Impact
1	Beta CDF wrong at every point	Beta(1,1).cdf(0.5) → 0.6534 (exact: 0.5); Beta(0.5,0.5).cdf(0.5) → −0.596, a negative probability
2	Incomplete gamma continued-fraction branch wrong	Affects Gamma.cdf and ChiSquare.cdf; errors to 0.26 absolute; ChiSquare(3).cdf(7.5) → 1.000498, a probability > 1
3	Beta.beta setter	Raises TypeError for every value — if alpha <= 0 sits outside its is not None guard
4	Binomial.p setter	Same defect pattern; raises TypeError for every value
5	DiscreteUniform.b setter	self.b = value instead of self._b → RecursionError
6	NegativeBinomial PMF overflow	inf at k=170, nan from k=200. C(k+r−1,k) via raw factorials; the coefficient itself is only 20301
7	Uniform setters skip cross-bound validation	Instance reaches a=10, b=−10, after which every method silently returns nan

#2 is the most serious for users — chi-square CDFs feed hypothesis tests, and
returning >1 corrupts them silently.

Run pytest -m known_bug -v to see them.

Notes for reviewers

• Contributors with existing clones should run
  git submodule deinit -f python/pybind11 after pulling
• Building now requires pybind11 installed (pip or system) rather than
  network access at configure time. pip install . handles this via
  build-system.requires; a direct cmake invocation needs
  pip install pybind11 first
• python_requires narrowed to >=3.12. 3.7–3.11 never actually worked

Follow-ups (not in scope here)

Fix the 7 defects above; MANIFEST.in (the sdist currently ships no C++ sources
and is unbuildable); py.typed and .pyi stubs; the PyPI name fastdist is
already taken; wire the C++ tests into ctest (they're built but never run, and
their assert()s are stripped under NDEBUG); CI matrix across 3.12–3.14 and
Linux/macOS/Windows.