numpy-level point-kind geo2mort: geo2mort(..., points=True)#100
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #100 +/- ##
==========================================
+ Coverage 93.70% 93.73% +0.03%
==========================================
Files 9 9
Lines 1223 1230 +7
==========================================
+ Hits 1146 1153 +7
Misses 77 77
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report in Codecov by Harness.
🚀 New features to boost your workflow:
|
Merging this PR will not alter performance
Comparing Footnotes
|
espg
left a comment
There was a problem hiding this comment.
🤖 from Claude (review)
Adversarial pass on the point-encoder. I built the extension (maturin develop --release) and exercised the behavior directly. The core encode is correct against every acceptance criterion in issue #96: points=True words differ from area at order 29 but decode to the same nested cell; clip2order coarsening is bit-identical at orders 0–28 (clip at 29 preserves kind, so the test's range(0, 29) cap is intentional and correct — including 29 would fail, which is right); a lone point survives common_ancestor unchanged; output is uint64/C-contiguous/same-shape; the uint64 coercion is correct including base cells 7–11 with bit 63 set; from_latlon(points=True) equivalence holds; and the ValueError message matches from_latlon verbatim. The fused Rust path and default-order flip are both what the maintainer signed off on. No correctness bug found.
The findings are all test-quality / robustness, plus one thing to surface:
test_nonfinite_handling_agrees_between_pathsis close to vacuous (biggest one). It compares only exception type, but empirically neither path raises on NaN/±inf — both silently emit a valid-looking non-zero word — so the assert passes without checking anything. It should assert decoded-nested equality (which does hold), and useexcept BaseException(a PyO3 panic surfaces asPanicException, aBaseException). Related: this quietly means a NaN fill value on the per-photon hot path encodes to a plausible word rather than the reserved0or a loud failure the issue mentioned — pre-existing area-path behavior, but worth a maintainer eye.- Dead
# noqa: BLE001— no ruff/bugbear is configured (§7), so it silences nothing and misleads. - Untested broadcast branch for
points=True(scalar + array), one of the three sub-paths inrust_geo2mort. - Default
order18→29 is a silent public behavior change — authorized, but flag it as breaking in the PR body/changelog; and the Python side hardcodes29where the siblings useMAX_ORDER.
None of these block the encode; (1) is the one I'd fix before merge.
Generated by Claude Code
| geo2mort(bad, 0.0, order=29, points=True) | ||
| except Exception as exc: # noqa: BLE001 | ||
| point_err = type(exc) | ||
| assert area_err == point_err, f"non-finite path mismatch for {bad}" |
There was a problem hiding this comment.
🤖 from Claude (review)
This is the acceptance criterion the issue cares most about ("non-finite lat/lon should fail loudly or produce the sentinel 0 — the two paths must agree"), but as written the test is close to vacuous. I built the extension and ran the actual behavior:
nonfinite nan points=False -> value 5764607523034234909
nonfinite nan points=True -> value 5764607523034234928
nonfinite inf points=False -> value 1152921504606847005
nonfinite -inf points=False -> value 10376293541461622813
So on both paths a non-finite input neither raises nor returns 0 — the healpix hash silently maps NaN/±inf to a real cell (NaN → nested 1152921504606846976) and packs a valid-looking non-zero word. Because nothing raises, area_err == point_err == None and the assert passes without checking anything meaningful.
Two concrete problems:
- It never verifies value agreement. The docstring promises the paths "resolve identically", but the test only compares whether-it-raised. I confirmed the area and point words for NaN decode to the same nested cell (
1152921504606846976both) — that's the property worth pinning. Assert decoded-nested equality (viarust_mort2nested), not just exception type. except Exceptionis the wrong net for the Rust boundary. If the hash ever panicked instead of returning a word, PyO3 surfaces it aspyo3_runtime.PanicException, which derives fromBaseException, notException— it would escape this handler and error the test rather than being compared.except BaseException(or asserting the value) is the robust form.
Separately, worth surfacing for a maintainer decision (not something to silently absorb): the issue said "0 remains the only reserved word, never a valid encode result," yet a NaN photon — a realistic ATL03 fill value on the per-photon hot path this PR targets — silently encodes to a plausible non-zero word on both the area and point paths. This is pre-existing area-path behavior and matches the "mirror the area path" directive, so it isn't a regression, but the current test actively obscures it.
Generated by Claude Code
| area_err = point_err = None | ||
| try: | ||
| geo2mort(bad, 0.0, order=29, points=False) | ||
| except Exception as exc: # noqa: BLE001 - we compare the failure mode |
There was a problem hiding this comment.
🤖 from Claude (review)
# noqa: BLE001 is dead here. Per CLAUDE.md §7 the only linters configured are flake8 mortie --select=E9,F63,F7,F82 (blocking) and a non-blocking --max-line-length=88 style pass — there is no ruff/flake8-bugbear, so BLE001 (a ruff code) is never emitted. The suppression silences nothing and reads as if a bugbear check exists. CLAUDE.md §4 also discourages adding # noqa to quiet warnings rather than fixing the cause. Drop the # noqa (and if you take the except BaseException suggestion on the assert below, the broad-except concern the code was pre-empting goes away too).
Generated by Claude Code
| assert out.flags["C_CONTIGUOUS"] | ||
| assert out.shape == _LATS.shape | ||
| # scalar in -> length-1 ndarray, matching the area path's always-ndarray return. | ||
| scalar = geo2mort(45.0, -122.0, points=True) |
There was a problem hiding this comment.
🤖 from Claude (review)
Coverage gap on the point path: the points=True branch in rust_geo2mort has three sub-branches — both-scalar, equal-length array, and broadcast (scalar-lat + array-lon, or vice-versa, via the lat_bcast/lon_bcast flags in lib.rs). This suite exercises the equal-length array path (_point()) and the both-scalar path (this line), but never the broadcast branch with points=True. A one-liner like geo2mort(45.0, _LONS, points=True) asserting shape == _LONS.shape and dtype uint64 would close it and guard the branch that geo2mort_point_scalar is threaded through. (Empty-input geo2mort([], [], points=True) also works — returns (0,) uint64 — and is worth a line if you want the boundary pinned.)
Generated by Claude Code
|
|
||
|
|
||
| def geo2mort(lats, lons, order=18): | ||
| def geo2mort(lats, lons, order=29, points=False): |
There was a problem hiding this comment.
🤖 from Claude (review)
The order=18 → order=29 default flip is an authorized change (the maintainer asked for it directly on the issue), so this is not an objection — two follow-through notes:
- It's a silent public behavior change for the bare
geo2mort(lat, lon)call: external callers relying on the old order-18 default now get a different (order-29) word. In-tree this is safe — I grepped every caller and all pass an explicitorder=except the docstring examples — but it should be called out explicitly in the PR description / changelog as a breaking behavior change, not just as "issue numpy-level point-kind geo2mort: geo2mort(lat, lon, order=29, points=True) #96". - Minor consistency: this function hardcodes the literal
29twice (theorder=Noneresolution on line 140 and the guard on line 141), whereas the siblingMortonIndexArray.from_latlonand the Rust guard both compare against theMAX_ORDERconstant. Not load-bearing (29 is the u64-packing ceiling and won't move), but a named constant would keep the two surfaces from drifting.
Generated by Claude Code
why don't we has to the reserved 0 ? we already have base cell '0' reserved for this... |
this seems fine.
agreed.
i did mean that. lan / lon inputs are generally expected to be of type point (i.e., indeterminate resolution, encoded at max precision). so... flip it |
espg
left a comment
There was a problem hiding this comment.
🤖 from Claude (review)
Fresh-eyes review of phase 4 (295113f) — the non-finite→reserved-0 and default-flip directives. Both directives are faithfully implemented; the change is correct. Findings are all non-blocking nits.
Verified correct
- Tri-state truth table — enumerated all 8 cases (bare / order=18 / order=29 / points=True / points=True+order=18 / points=False / points=False+order=18 / order=0). All resolve as intended. Critically, the
order=0falsy trap is avoided: the code usesorder is None(identity), notorder or MAX_ORDER(truthiness), soorder=0stays0and inferspoints=False— an order-0 area cell, not a coerced order-29 word. - Non-finite guard — correct for NaN/±inf in either lat or lon, on both routes, and correctly scoped: living in
geo2mort_word(notgeo2mort_scalar) leaveslinestring's internal hashing unchanged (linestring.rs:126still calls the scalar directly). Array path zeroes only the bad rows (tested), empty array still fine.0is genuinely unreachable by a real encode (prefix is base+1, so the empty sentinel never collides). - CHANGELOG "Breaking" claim — "was order-18 area cells" is accurate against the last release (0.8.4 defaulted
order=18); the intermediate order-29-area default was itself unreleased. The[Unreleased]block has a single, non-contradictory line. - Point/area/broadcast/dtype/from_latlon-equivalence and the
points=True, order=18raise all remain covered.
Non-blocking nits (inline)
- No test pins that a falsy
order=0yields an order-0 word — the backward-compat test comparesorder=0toorder=0, so a futureorder or MAX_ORDERregression would slip past. A one-lineinfer_order_from_morton(...) == 0assert would lock it down. (Highest-value follow-up.) rust_geo2mort's pyo3 signature default (order=29, points=false) and docstring now describe an internal-only default that diverges from the publicgeo2mortpoint default — harmless (Python always passes explicit args) but a drift footgun for direct-binding callers.
Nothing here blocks the phase.
Generated by Claude Code
| # Resolve the point/area mode: a bare call encodes points; an explicit order | ||
| # implies an area cell at that resolution unless the caller forces points. | ||
| if points is None: | ||
| points = order is None |
There was a problem hiding this comment.
🤖 from Claude (review)
Tri-state resolution is correct, and I want to explicitly flag the order=0 falsy trap since it's the easiest place to have introduced a bug — you avoided it. Because this uses the identity test order is None (not a truthiness test like order or MAX_ORDER), a falsy-but-valid order=0 is handled right:
geo2mort(lat, lon, order=0)→points = (0 is None) = False,orderstays0→ order-0 area cell. ✔geo2mort(lat, lon, order=0, points=True)→ raises (0 != 29). ✔ (tested)
Enumerated the full truth table (bare / 18 / 29 / points=True / points=True,order=18 / points=False / points=False,order=18 / order=0) — all resolve as intended.
One residual gap: nothing in the suite actually pins that order=0 produces an order-0 word. test_area_default_backward_compatible_at_explicit_order iterates order=0 but only compares order=0 against order=0, points=False, so a hypothetical future order or MAX_ORDER regression (0 → 29) would keep both sides equal and slip past. A one-line assert infer_order_from_morton(int(geo2mort(_LATS, _LONS, order=0)[0])) == 0 would lock the falsy-order behavior down. Non-blocking.
Generated by Claude Code
| np.testing.assert_array_equal( | ||
| geo2mort(_LATS, _LONS, order=12), | ||
| geo2mort(_LATS, _LONS, order=12, points=False), | ||
| ) |
There was a problem hiding this comment.
🤖 from Claude (review)
Good coverage of the flipped default here (bare == points=True, explicit order == area). The one thing not asserted anywhere is that an explicit falsy order=0 stays order 0 rather than being coerced to 29 — see my note on tools.py. Since order=0 is now the only falsy-but-valid order flowing through the order is None gate, it's worth an explicit pin (infer_order_from_morton(...) == 0) so the tri-state logic can't silently regress. Non-blocking.
Generated by Claude Code
| // Both scalars → return scalar | ||
| if lats_is_scalar && lons_is_scalar { | ||
| let result = geo2mort::geo2mort_scalar(lat_arr[0], lon_arr[0], order); | ||
| let result = geo2mort::geo2mort_word(lat_arr[0], lon_arr[0], order, points); |
There was a problem hiding this comment.
🤖 from Claude (review)
Routing both the scalar and array paths through geo2mort_word is a clean dedup, and scoping the non-finite→0 guard inside geo2mort_word (rather than geo2mort_scalar) correctly leaves linestring's internal hashing untouched — verified linestring.rs:126 still calls geo2mort_scalar directly. ✔
Minor drift note (non-blocking): this pyfunction still carries #[pyo3(signature = (lats, lons, order=29, points=false))] and the docstring says "order (default 29)". All of the new point/area/tri-state policy now lives in Python (tools.py always passes both args explicitly), so this Rust-level default is dead for the public surface — but a direct _rustie.rust_geo2mort(lat, lon) still yields an order-29 area word, i.e. the pre-flip behavior. Harmless today, but it's a documentation/behavior-drift footgun if anyone later calls the binding directly. Consider a one-line comment noting the binding defaults are internal-only and the public default lives in geo2mort.
Generated by Claude Code
|
🤖 from Claude Both landed in phase 4 (self-reviewed + folded in phase 5):
Truth table now: bare → point@29; Generated by Claude Code |
Closes #96.
What this does
Adds a vectorized, numpy-in/numpy-out point-kind encoder on the
geo2mortsurface, so zagg's hot path (HealpixGrid.assign, per-photon for ATL03) gets a public vectorized point encode and drops theMortonIndexArray→ ndarray unwrap entirely (the ~12x penalty measured in the issue).lat/lon inputs are treated as points by default (indeterminate resolution, encoded at max precision), so:
Result is always a plain contiguous
uint64ndarray — no ExtensionArray wrapper. Point encoding is order-29-only, sopoints=Truewith an explicitorder != 29raisesValueError(matchingMortonIndexArray.from_latlon). Non-finitelat/lonencode to the reserved empty word0(base cell 0 is the null sentinel) on both routes.Recorded design decisions (from review thread)
pointsdefaults to point. Per this comment ("lat/lon inputs are generally expected to be of type point ... flip it"), a baregeo2mort(lat, lon)returns order-29 point words. An explicitorderimplies an area cell (points inferredFalse), sogeo2mort(lat, lon, order=18)does not raise. Resolved via a tri-stateorder=None/points=None:points = order is None, thenorder = order or 29— using an identity check soorder=0stays 0.0. Per this comment ("why don't we hash to the reserved 0? we already have base cell '0' reserved for this"), non-finite lat/lon map to0on both paths. The guard lives in a newgeo2mort_wordhelper scoped to the publicgeo2mortsurface, solinestring's internalgeo2mort_scalarhashing is unchanged.CHANGELOG.md. All in-tree callers pass an explicitorder.Phases
geo2mort_point_scalar;rust_geo2mortgainspointsflag; Rust unit tests (point ≠ area at 29 but same nested cell; fused == compose).geo2mortpoints/order plumbing + validation + uint64 coercion; the four issue-specified tests +from_latlonequivalence.noqaremoval, broadcast test,MAX_ORDERconstant.0(geo2mort_word, scoped to the public surface, + Rust test); flip thepointsdefault to point-by-default via the tri-state resolution; CHANGELOG line; updated/added tests (reserved-0, bare-call-defaults-to-point).order=0resolves to an order-0 word (guards against a future truthiness regression); clarify the binding's low-level-default doc comment.How it was tested
cargo test(13 geo2mort tests),cargo fmt --checkclean,cargo clippy(no new warnings — only the pre-existinguseless_conversionfalse-positives, none in this diff).flake8 mortie --select=E9,F63,F7,F82clean.pytest -v: 548 passed, 11 skipped.test_geo2mort_points.pycovers the four issue tests,from_latlon/kernel equivalence, broadcast, reserved-0 non-finite handling, bare-call-defaults-to-point, and pins the explicit-order (incl.order=0) area path.Side note: the scalar path previously returned
int64for small (northern) words anduint64for large (bit-63-set southern) words — a latent hemisphere-dependent dtype inconsistency.geo2mortnow always returns a contiguousuint64ndarray.Questions for review
None outstanding — both earlier questions were answered on the thread and implemented in phase 4. Ready for your review.