Standardize test conventions and modernize docstring/lint tooling

.github/workflows/alltests.yml

@@ -312,6 +312,15 @@ jobs:
         run: |
           make check_colab_notebooks
           make check_colab_notebooks_smoke
+      - name: Check test-suite conventions (Linux)
+        if: runner.os == 'Linux'
+        shell: bash -l {0}
+        run: |
+          set -e
+          make check_test_style STRICT=--strict
+          python -m pip install -q "pydoclint>=0.5.0"
+          make check_docstring   # informational (exits 0 without STRICT)
+          make check_baseline    # real gate: docstring/pydoclint/annotation issues must not increase
       # -----------------------------------------------------------
       # Install minimal LaTeX required by Jupyter notebooks (OS-specific)
       # -----------------------------------------------------------

CONTRIBUTING.md

@@ -163,6 +163,10 @@ make tests
 
 Please see the targets in the makefile for more granular control over tests.
 
+### Test file layout
+
+Unit tests live flat in `test/`, named `test_<area>_<topic>.py` where `<area>` is a short code for the `qmcpy` subpackage under test (`tm` true_measure, `dd` discrete_distribution, `sc` stopping_criterion, `ig` integrand, ...) or a cross-cutting bucket (`ee`, `sr`). So `pytest test/ -k test_tm_` runs every true-measure test. A test that spans two areas goes under the component actually under test, with the other named in `<topic>` (e.g. `test_sc_cubbayes_kernels.py`); use `ee` only when neither side is the clear subject, and never coin a new code — `STRICT=--strict` rejects anything outside the table. New files should also be written as a `unittest.TestCase` subclass rather than bare `def test_*` functions. `make check_test_style` lists any file that breaks either convention (informational; also runs inside `make format`; `STRICT=--strict` makes it fail). The full area table is in [`test/README.md`](test/README.md#test-file-organization).
+
 ## Documentation
 
 ### Ensure `pyreverse` Is On Your PATH

docs/api/discrete_distributions.md

@@ -70,10 +70,7 @@ python -m pip install "qmcpy[mpmc]"
 qmcpy-install-mpmc
 ```
 
-The second command selects the `pyg_lib` wheel page matching the installed
-PyTorch and accelerator builds. For GPU support or platform-specific wheels,
-see the [PyTorch installation guide](https://pytorch.org/get-started/locally/)
-and the [PyTorch Geometric installation guide](https://pytorch-geometric.readthedocs.io/en/latest/install/installation.html).
+The second command selects the `pyg_lib` wheel page matching the installed PyTorch and accelerator builds. For GPU support or platform-specific wheels, see the [PyTorch installation guide](https://pytorch.org/get-started/locally/) and the [PyTorch Geometric installation guide](https://pytorch-geometric.readthedocs.io/en/latest/install/installation.html).
 
 ::: qmcpy.discrete_distribution.mpmc.mpmc.MPMC
 

docs/good_practices.md

@@ -38,10 +38,27 @@ When notebook-backed content changes:
 QMCPy documentation is built from docstrings, so public APIs should document their behavior clearly and consistently.
 
 - Use **Google-style docstrings** for public classes, methods, and functions.
-- Document parameters, return values, shapes, assumptions, and any stochastic behavior.
+- Start every docstring with a one-line summary before any section header.
+- Document every parameter and return value, plus shapes, assumptions, and any stochastic behavior. Constructor arguments go in the `__init__` method's own docstring, with the type in the docstring (`name (type): ...`).
+- Put a blank line before every section header (`Args:`, `Returns:`, `Raises:`, `Examples:`, ...) and write the header as `Name:` — not a NumPy-style `Name` followed by an `-----` underline.
 - Include short doctestable examples when they clarify expected use.
 - Update docstrings at the same time as the implementation so the rendered API docs do not drift from the code.
 
+`make check_docstring` runs two checks over public objects under `qmcpy/`:
+
+- `scripts/check_docstring.py` for **formatting** — a one-line summary before the first section (`missing-summary`), no NumPy-style sections, a blank line before every section header, canonical `Name:` headers, and public objects with no docstring. After the overall count it prints a second summary restricted to files changed relative to `DOCSTRING_BASE` (default `develop`), so you can see your branch's contribution to the backlog.
+- `pydoclint` (configured in `pyproject.toml` under `[tool.pydoclint]`) for **content** — every parameter and return value is documented and matches the signature, in Google form.
+
+It is informational by default; `STRICT=--strict make check_docstring` makes both parts fail the build. Pass `CHECK_DOCSTRING_ARGS=--skip-missing` to skip the "no docstring" formatting check, or `DOCSTRING_PATH=qmcpy/true_measure` to narrow the scan. `make check_docstring_changed` runs the same two checks on just the `qmcpy/*.py` files that changed relative to `DOCSTRING_BASE` — the quick check to run before opening a PR (it is also part of `make format`).
+
+For annotated public APIs, `make add_docstring_arg_types` inserts missing Google-style argument types into existing `Args:` entries from the function signature. For example, `distance: float` becomes `distance (float): ...` in the docstring. Use `DOCSTRING_TYPE_PATH=path/to/file.py` to narrow the scan, or run `make add_docstring_arg_types_changed` to apply it only to Python files reported by `git diff --name-only develop -- '*.py'`. Use `DOCSTRING_TYPE_DIFF_BASE=origin/develop` to compare against a different base, and use `make check_docstring_arg_types_changed` to fail when changed files still need annotation-derived updates. The helper does not infer types for unannotated functions and does not invent missing scientific argument descriptions.
+
+For changed public APIs, `make annotate_public_api_types_changed` performs the reverse operation conservatively: it copies explicit, valid Google `Args:` and `Returns:` types into missing function annotations and adds `-> None` to constructors. It never replaces an existing annotation. Types that are prose, use syntax unsafe for Python 3.9, or reference names not already available in the module are reported and skipped. Then `make sync_docstring_types_changed` copies the resulting signature annotations back into existing `Args:`, `Returns:`, and `Yields:` descriptions. Run the annotation target before the synchronization target, review the complete diff, and run `make check_public_api_types_changed` for a non-mutating consistency check. All three targets default to files under `qmcpy/` changed relative to `develop`; override this with `PUBLIC_API_TYPE_PATH` or `PUBLIC_API_TYPE_DIFF_BASE`.
+
+These helpers synchronize explicit type information; they do not infer a scientific API contract from default values, implementation expressions, or one observed runtime type. They also do not invent missing docstring descriptions or sections. Resolve every reported conflict manually, especially scalar-versus-array inputs, optional values, shape conventions, and abstract interfaces.
+
+For mostly well-formed Google-style docstrings, developers may also use the optional open-source `format-docstring` helper to normalize wrapping and existing argument type syntax. Install it locally with `python -m pip install format-docstring`, then run `make format_google_docstrings` to apply it under `qmcpy/`, or run `make format_google_docstrings_changed` to apply it only to Python files reported by `git diff --name-only develop -- '*.py'`. Always review the resulting diff because automated formatting can reflow examples and prose.
+
 ## Extend the Existing Object Model
 
 New functionality should fit the existing QMCPy class hierarchy instead of introducing parallel designs without discussion.
@@ -82,6 +99,10 @@ Several reviews focused on avoidable cleanup that is easy to catch before reques
 - Remove unused imports, trailing whitespace, and other style-only churn before requesting review.
 - Use explicit runtime exceptions such as `ParameterError` for invalid user inputs instead of relying on `assert` statements in production code.
 
+For a mechanical first pass, `make check_asserts_changed` reports standalone assertions in production Python files changed relative to `ASSERT_DIFF_BASE` (default `develop`) and returns nonzero when conversions are available. `make convert_asserts_changed` uses the open-source [LibCST](https://libcst.readthedocs.io/) codemod library to convert those assertions to explicit `AssertionError` raises while preserving comments and formatting. Use `make convert_asserts ASSERT_PATH=path/to/file.py` for a specific file or directory.
+
+`AssertionError` is the conservative default because it preserves the original exception class and message while making validation active under `python -O`. For a reviewed set of input checks, a developer may select an exception already imported by every target file, for example `make convert_asserts ASSERT_PATH=path/to/file.py ASSERT_EXCEPTION=ParameterError`. The tool does not infer whether a condition represents invalid input, a dimension mismatch, or an internal invariant; choose `ParameterError`, `DimensionError`, `ValueError`, or another public exception only after reviewing the API contract. Assertions sharing a semicolon-delimited line with another statement, or appearing in a one-line compound suite such as `if condition: assert invariant`, are reported but skipped. Always inspect the complete diff and run the focused tests after conversion.
+
 ## Add Demos or Blogs as Notebooks
 
 User-facing methods, new workflows, and mathematically important additions should usually come with an executable notebook.

docs/tests.md

@@ -25,6 +25,52 @@ This document describes the available test targets in the Makefile for QMCSoftwa
 | `make delcoverage` | Reset coverage tracking | Instant | Start fresh coverage analysis |
 
 
+## Test File Organization
+
+Unit tests live flat in `test/` (no subpackage subfolders). Every file is named:
+
+```
+test_<area>_<topic>.py
+```
+
+`<area>` is a short code for the `qmcpy` subpackage under test, or a cross-cutting bucket:
+
+| area | scope |
+|------|-------|
+| `dd` | `qmcpy/discrete_distribution` |
+| `ft` | `qmcpy/fast_transform` |
+| `ig` | `qmcpy/integrand` |
+| `kn` | `qmcpy/kernel` |
+| `sc` | `qmcpy/stopping_criterion` |
+| `tm` | `qmcpy/true_measure` |
+| `ut` | `qmcpy/util` |
+| `ee` | end-to-end / cross-cutting pipeline (`integrate()`, worked problems such as Keister and pi) |
+| `sr` | `scripts/` tooling, packaging, and docs checks |
+
+This keeps related tests adjacent when the directory is sorted, and lets you run one area at a time:
+
+```bash
+python -m pytest test/ -k test_tm_      # every true_measure test
+make unittests PYTEST_EXTRA_ARGS="-k test_sc_"
+```
+
+When a test spans two areas (say a stopping criterion exercised against a particular kernel), file it under the component actually under test and name the other in `<topic>` — e.g. `test_sc_cubbayes_kernels.py`. Reserve `ee` for cases where neither side is the clear subject. Do not invent new area codes: only the prefixes in the table are accepted, and `make check_test_style STRICT=--strict` fails on anything else.
+
+Notebook tests are separate: they live in `test/booktests/` as `tb_*.py` and are generated from `demos/` (see `test/booktests/README.md`).
+
+### Conventions checked by `make check_test_style`
+
+1. **Area prefix** — the filename must start with a recognized `test_<area>_` prefix from the table above.
+2. **Object class** — write a test file as one or more `unittest.TestCase` subclasses rather than bare `def test_*` pytest functions. A class groups related assertions under a name (so `pytest -k TestCubMCG` selects them and a failure report names the group), shares construction through `setUp` / `setUpClass` / `self.addCleanup`, and runs identically under `pytest`, `python -m unittest`, and the coverage and booktest runners without depending on pytest fixtures. Most of the suite already follows this; a few legacy files still use bare functions and new files should not.
+
+`make check_test_style` lists any violation and is informational (exit 0). It also runs as part of `make format`. To make it fail instead — for a pre-commit hook or CI gate — pass `--strict`:
+
+```bash
+STRICT=--strict make check_test_style
+```
+
+`STRICT=--strict make check_test_style` also runs in CI (the `alltests` workflow), so both conventions are enforced on every pull request.
+
 ## Detailed Descriptions
 
 ## Scope
@@ -153,6 +199,7 @@ Runs notebook tests with **Parsl distributed parallelization** for compute-heavy
 - **Dependencies**: Parsl must be installed and configured
 - **Use when**: Running large notebook suites with distributed compute resources
 
+
 ---
 
 ### Helper / Internal Targets
@@ -242,7 +289,6 @@ Displays the current coverage report (must run other targets first to accumulate
 Deletes `.coverage` and `coverage.json` files to reset coverage tracking.
 - **Use before**: Running a fresh coverage report without accumulated data
 
-
 ---
 
 ## Currently Active Targets: Justification