dimos map tooling, stream alignment by leshy · Pull Request #2306 · dimensionalOS/dimos

leshy · 2026-05-29T12:31:03Z

`dimos map` utilities for dataset collection team

you can ignore those, they are temporary,

changing pose source for observation stream, rendering global map, loop closures etc. lots of tooling (temporary and will change a lot)

dataset validation instructions document

stream alignment

a very simple stream.align tool - this needs much more work. multi stream input and output, interpolation of poses/transforms etc

Replaces the monolithic pgo_then_voxels with four primitives over mem2 Streams and Transforms: pgo_keyframes(lidar) -> Stream[Keyframe] keyframes_to_corrections(kfs) -> Stream[Transform] (world_corrected <- world_raw) make_interpolator(corrections) -> (ts) -> Transform (SLERP + linear, endpoint-clipped) apply_corrections(stream, corr) -> Stream[T] (shuffles obs.pose) Drift correction is now a first-class, reusable Stream[Transform] that any pose-stamped consumer can apply — same math as before (rigid T_corr = T_global @ T_local^-1 per keyframe, SLERP + linear interpolation between bracketing keyframes), just composable. dimos map --pgo migrates to the new primitives; pgo_then_voxels is deleted. The internal _SimplePGO / PGOConfig / _KeyPose machinery stays in pgo.py and is imported by pgo2.

Renames: pgo.py -> pgo_internals.py (gtsam/ICP machinery) pgo2.py -> pgo.py (public Stream-shaped API) `dimos.mapping.relocalization.pgo` is now the canonical import path (`pgo_keyframes`, `keyframes_to_corrections`, `make_interpolator`, `apply_corrections`, `correction_at`, `Keyframe`). The internal _SimplePGO / PGOConfig / _KeyPose / _icp / _voxel_downsample helpers live in pgo_internals.py and are imported lazily inside pgo.py to keep gtsam off the public-API import path.

Adds test_pgo.py covering pose normalization on Observation, the Transform↔Pose3 conversion helpers, the interpolator edge cases, apply_corrections behavior, and a real-recording smoke test (skipped when data/go2_short.db is absent). Annotates the two map.py helpers so they pass mypy. Removes leftover section divider comments to satisfy the no-section-markers project rule.

The colors-copy block in `PointCloud2.transform` calls `self.pointcloud` to check `has_colors()`, which forces a tensor->legacy conversion on every invocation. With `pgo --full-pgo` rebuilding from hundreds of lidar frames, that hidden allocation dominates the per-frame cost. The colors path was lidar-irrelevant (lidar clouds have no colors anyway) and the feature it was meant to support never landed; remove it so transform() stays a clean numpy round-trip.

…s into feat/ivan/pgo_rewrite

Group the map CLI scripts (globalmap, pose_fill, rename, replay, replay_marker, summary) plus their assets and dataset_validation doc under a cli/ subpackage. distance.py stays in utils/ as a shared lib. The map CLI test moves alongside as cli/test_cli.py.

These files were inadvertently diverged on this branch. Restore both to main; the map-marker CLI callers remain compatible with main's DetectMarkers API.

These end-to-end CLI tests forked python -m dimos.robot.cli.dimos per case, re-paying the heavy per-verb imports (rerun, open3d, voxel) 7 times. Run them in-process via Typer's CliRunner so those imports are paid once: 33.7s -> 7.5s, all 7 still pass. Trade-off: shared interpreter, no per-test process isolation.

Add from_time/to_time (relative to the first observation) and from_timestamp/to_timestamp (absolute epoch seconds) for windowing a stream by time. A trailing to_time is a duration measured from the current start, so from_time(2).to_time(30) reads as "skip 2s, take the following 30s"; frames mix freely (from_timestamp(ts).to_time(30)). Shared base for the stream-alignment (#2306) and go2dds (#2314) branches, which both need this windowing API.

…arch - pgo_keyframes: widen stream param to Iterable[Observation[PointCloud2]] (clears the mypy gate; it never uses Stream methods) - pose_fill_db: order_by("ts") on both align inputs — align requires ts-ascending iteration, but sqlite defaults to id ASC (insertion order), which would silently mis-pair out-of-order recordings - add CI-running unit tests for Observation.with_pose (lazy payload kept) and pose_fill(mount=...) composition - remove the unreferenced autoresearch/ research drop (near-duplicate of pgo_auto.py + scaffolding); pgo_auto.py kept

Add from_time/to_time (relative to the first observation) and from_timestamp/to_timestamp (absolute epoch seconds) for windowing a stream by time. A trailing to_time is a duration measured from the current start, so from_time(2).to_time(30) reads as "skip 2s, take the following 30s"; frames mix freely (from_timestamp(ts).to_time(30)). Shared base for the stream-alignment (#2306) and go2dds (#2314) branches, which both need this windowing API.

Pulls the stairs dataset LFS pointer from feat/ivan/go2stairs so the map tooling can use it; object already on remote LFS store.

…nment # Conflicts: # dimos/memory2/stream.py # dimos/memory2/test_stream.py

The self-hosted CI image's published :dev tag is stale (predates libturbojpeg0-dev in docker/python/Dockerfile), so TurboJPEG() raises at runtime and the four verbs that decode the color_image stream fail. Guard them with skipif so they skip on a lib-less image and still run where the native lib is present.

…nment

paul-nechifor · 2026-06-01T22:12:10Z

+    if no_gui:
+        print(f"open with: rerun {out}")
+    else:
+        subprocess.Popen(["rerun", str(out)])


You maintain to reference to this? How is it ever closed?

paul-nechifor · 2026-06-01T22:16:59Z

+        --out mid360_renamed.db \\
+        --rename go2_lidar=lidar \\
+        --rename lidar=fastlio_lidar \\
+        --rename odometry=fastlio_odometry


This is meant to be used through dimos map rename, no? Then please don't document running the file itself.

paul-nechifor · 2026-06-01T22:35:22Z

+
+    for p in primary_iter:
+        # Advance the cursor until `nxt` sits past `p`; `prev` trails just behind.
+        while nxt is not None and nxt.ts <= p.ts:


Why <=? If nxt.ts == p.ts isn't that the best possible solution?

paul-nechifor · 2026-06-01T22:42:47Z

+            pose=pose,
+            _data=_UNLOADED,
+            _loader=lambda: self.data,
+            _data_lock=threading.Lock(),


Suggested change

_data_lock=threading.Lock(),

Already the default. (But it's a bit odd to pass locks around. Usually synchronization should be owned by the object protecting the state, not having an external call telling it what to synchronize on.)

paul-nechifor · 2026-06-01T22:43:56Z

+"""Print ``Store.summary()`` for a memory2 sqlite recording.
+
+Usage:
+    uv run python -m dimos.mapping.utils.cli.summary mid360


Suggested change

uv run python -m dimos.mapping.utils.cli.summary mid360

uv run dimos map summary mid360

paul-nechifor · 2026-06-01T22:54:07Z

+        # Skip placeholder poses (origin position OR zero quaternion).
+        if pose[0] == 0 and pose[1] == 0 and pose[2] == 0:
+            continue
+        if pose[3] == 0 and pose[4] == 0 and pose[5] == 0 and (pose[6] == 0 or pose[6] == 1):
+            continue


Valid values shouldn't be placeholder values. I don't see why valid values like this should be skipped? Can't you use None (an invalid pose) as a placeholder?

leshy added 30 commits May 24, 2026 18:53

map tool first pass

ceb08ba

map tool wrap

4e0cec3

renamed rerun blueprint to dimos map tool

2099e00

readme cleanup

47848d4

pgo rewrite

25cf609

pgo cleanup

45e72da

visualizing loop closures

df4435f

pgo vis fixed

ebdf1ec

much faster pgo

71762c1

Merge branch 'main' into feat/ivan/pgo_rewrite

74a5011

ci: install gtsam-extended in tests group so import gtsam works

626a4fc

Merge branch 'main' into feat/ivan/pgo_rewrite

43c0453

markers on dimos map

2f46b38

map detects markers correctly

82e2dab

maptool speed limits

b98b506

Merge branch 'feat/ivan/pgo_rewrite' of github.com:dimensionalOS/dimo…

c1115d8

…s into feat/ivan/pgo_rewrite

markers_rrd renderer

4d5e9b6

average window for markers

b3163bc

sliding window 7.5s

e529913

detector tuned

5751d48

nicer map

dcd7f7b

marker eval first pass

a6d4dcc

extracted loop closure code from relocalization code

89eb996

Merge branch 'autoresearch/ivan/pgo_go2' into feat/ivan/pgo_rewrite

22ff976

better eval

186a4be

Merge branch 'autoresearch/ivan/pgo_go2' into feat/ivan/pgo_rewrite

0d8b277

greptile-apps Bot reviewed May 30, 2026

View reviewed changes

Comment thread dimos/memory2/module.py

leshy added 3 commits May 30, 2026 18:05

Revert branch-local changes to marker.py and marker_transformer.py

c91992c

These files were inadvertently diverged on this branch. Restore both to main; the map-marker CLI callers remain compatible with main's DetectMarkers API.

Remove unused loop_closure/utils/autotrim.py

3721fda

map cli: rename globalmap.py to map.py

6c31c4e

greptile-apps Bot reviewed May 30, 2026

View reviewed changes

Comment thread dimos/memory2/stream.py Outdated

leshy changed the title ~~Feat/ivan/stream alignment~~ dimos map tooling, stream alignment, autoresearch sketch Jun 1, 2026

leshy added the PlzReview label Jun 1, 2026

leshy mentioned this pull request Jun 1, 2026

GO2 MCAP/DDS integration as memory2 backend #2314

Open

leshy mentioned this pull request Jun 1, 2026

memory2: time windowing on Stream #2315

Merged

cleanup

501f3e3

greptile-apps Bot reviewed Jun 1, 2026

View reviewed changes

Comment thread dimos/memory2/stream.py Outdated

leshy changed the title ~~dimos map tooling, stream alignment, autoresearch sketch~~ dimos map tooling, stream alignment Jun 1, 2026

greptile-apps Bot reviewed Jun 1, 2026

View reviewed changes

Comment thread dimos/robot/cli/dimos.py

data: add go2_mid360_stairs recording from go2stairs branch

c70598b

Pulls the stairs dataset LFS pointer from feat/ivan/go2stairs so the map tooling can use it; object already on remote LFS store.

greptile-apps Bot reviewed Jun 1, 2026

View reviewed changes

Comment thread dimos/mapping/loop_closure/pgo_auto.py

Merge remote-tracking branch 'origin/main' into feat/ivan/stream_alig…

ffb412e

…nment # Conflicts: # dimos/memory2/stream.py # dimos/memory2/test_stream.py

leshy mentioned this pull request Jun 1, 2026

Feat/ivan/go2stairs #2283

Closed

leshy added 2 commits June 1, 2026 20:51

Merge remote-tracking branch 'origin/main' into feat/ivan/stream_alig…

ae17740

…nment

paul-nechifor reviewed Jun 1, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

dimos map tooling, stream alignment#2306

dimos map tooling, stream alignment#2306
leshy wants to merge 77 commits into
mainfrom
feat/ivan/stream_alignment

leshy commented May 29, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

paul-nechifor Jun 1, 2026

Uh oh!

paul-nechifor Jun 1, 2026

Uh oh!

paul-nechifor Jun 1, 2026

Uh oh!

paul-nechifor Jun 1, 2026

Uh oh!

paul-nechifor Jun 1, 2026

Uh oh!

paul-nechifor Jun 1, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	uv run python -m dimos.mapping.utils.cli.summary mid360
	uv run dimos map summary mid360

Conversation

leshy commented May 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

dimos map utilities for dataset collection team

stream alignment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

paul-nechifor Jun 1, 2026

Choose a reason for hiding this comment

Uh oh!

paul-nechifor Jun 1, 2026

Choose a reason for hiding this comment

Uh oh!

paul-nechifor Jun 1, 2026

Choose a reason for hiding this comment

Uh oh!

paul-nechifor Jun 1, 2026

Choose a reason for hiding this comment

Uh oh!

paul-nechifor Jun 1, 2026

Choose a reason for hiding this comment

Uh oh!

paul-nechifor Jun 1, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

leshy commented May 29, 2026 •

edited

Loading

`dimos map` utilities for dataset collection team