fix(ceiling): NaN for non-positive split-half reliability, fail loudly on an unsplittable split

[LeonHafner]

Summary

Hardens the data ceiling merged in #243. The Spearman-Brown correction was applied unconditionally, so a non-positive split-half reliability produced a negative or infinite "ceiling" instead of no answer, and _disjoint_halves failed obscurely at two edges.

No version bump: 0.8.2 is unreleased, so these fixes ship as part of it.

1. Spearman-Brown below zero

r' = 2r/(1+r) is a reliability correction only for r > 0. Below zero it is a pole, and it was applied to any value:

measured r	ceiling before	now
-0.5	-2.0	NaN
-0.9	-18.0	NaN
-1.0	-inf	NaN
0.0	0.0	NaN
0.5	0.667	0.667

Note the r = -1 case is a silent -inf, not a crash: polars renders float division by zero as infinity rather than raising, so a wrong number reached agg_ceiling_results.csv with no error.

Three metrics in SB_METRICS are sign-unbounded and can land there on a small or degenerate context — precisely when someone reaches for a ceiling: pearson_delta, de_spearman_sig, de_spearman_lfc_sig.

A non-positive split-half reliability means the halves carry no shared signal, i.e. there is no defensible ceiling — so it is reported as NaN, the same disposition already used for metrics excluded from the correction. A negative value would instead claim a ceiling worse than any achievable score. Null means fall through the same branch.

The threshold is 0 for every metric, including pr_auc / roc_auc whose chance baseline is 0.5 rather than 0 — a below-chance AUC is still corrected. That is deliberate: a 0.5 floor would be a stricter rule than the one the ceiling was empirically validated under. Recorded in the docstring so it stays a conscious choice.

2. _disjoint_halves failed obscurely at both edges

• Nothing splittable (every perturbation has 1 cell) reached np.concatenate([]) and surfaced a bare numpy error.
• The control specifically having < 2 cells was silently dropped, so the failure appeared later in the pipeline as a missing control.

Both now raise a ValueError naming the cause. Perturbations dropped for having < 2 cells are counted and logger.warning-ed, since the ceiling is then averaged over a different perturbation set than the main evaluation's aggregate — worth surfacing rather than leaving to be discovered.

3. Documentation

• floor(n/2) rather than n/2 (three places), and an explicit note that an odd n discards the leftover cell — both halves must be the same depth for the doubling to hold.
• Cost recorded: the halves are materialized as copies, so peak memory is roughly 2x the real matrix on top of the already-loaded pair, and DE is computed for both halves, so a ceiling run roughly doubles wall time.
• --ceiling help text and docstring now state that the halves need their own DE, so a precomputed --de-real / --de-pred does not carry over to the ceiling.

Tests

• Guard: r = -1 / -0.9 / -0.5 and the r = 0 boundary all yield NaN — never a negative ceiling, an inf, or a raise; r > 0 still corrected; a null mean → NaN.
• Both new errors: nothing splittable, and a control with < 2 cells.
• End-to-end assertion is now two-sided over every SB metric. The previous assert np.all(cv <= 1.0 + 1e-9) was one-sided, so a -18.0 ceiling passed. Verified the new bound is not vacuous — all four SB metrics in the anndata profile return positive values, so it is actually exercised.
• Dropped a stale best_value == ONE docstring reference; SB_METRICS is an explicit inclusion list.

Verification

All four CI gates locally: ruff format --check clean, cell-eval --help OK, pytest 48 passed. ty check reports 3 diagnostics, all pre-existing on main (_baseline.py, _cli/_prep.py, a tutorial notebook) — none in the files touched here, and the count is identical with this branch's changes stashed.

[gemini-code-assist]

Code Review

This pull request refines the data ceiling computation by using floor(n/2) cells for disjoint splits, handling edge cases where perturbations or controls have fewer than two cells, and guarding the Spearman-Brown correction to only apply when the measured reliability is greater than zero (returning NaN otherwise to avoid poles). It also adds corresponding unit tests. The reviewer suggested a performance optimization in _disjoint_halves to call .unique() on the perturbation column before casting to strings, which avoids an inefficient O(N) operation on large datasets.