Feat/ivan/go2stairs

[leshy]

forked from crappy branch just to give you this map_rrd thing, no need for actual work, feel free to copy just the db file

cd dimos/mapping/loop_closure/utils
python map_rrd.py go2_mid360_stairs --map --out stairs.rrd && rerun stairs.rrd

[greptile-apps]

Greptile Summary

Adds map_rrd.py, a standalone utility that reads a recorded SQLite dataset and emits a Rerun .rrd file containing lidar clouds, optional accumulated voxel maps, odometry trajectories, and camera frames. A handful of supporting changes land alongside it.

• PointCloud2.to_rerun now normalises the height colormap to the 1st–99th percentile range instead of global min/max, preventing distant outliers from washing out the colour scale.
• StreamAccessor gains __contains__ and __iter__, enabling the idiomatic "stream_name" in store.streams checks used throughout map_rrd.py, and Store.summary() is added as a convenience one-liner.
• resolve_named_path accepts a new pull=False keyword to skip LFS download and fail fast instead.

Confidence Score: 4/5

Safe to merge after fixing the broken as_numpy() unpacking in the test.

The test in test_lcm_encode_decode now assigns the full (points, colors) tuple to original_points instead of unpacking it, so the point-count assertion trivially passes and assert_allclose compares wrong objects. All other changes — the new map_rrd.py script, colormap percentile clipping, StreamAccessor additions, and resolve_named_path keyword — look correct.

dimos/msgs/sensor_msgs/test_PointCloud2.py — the as_numpy() unpacking fix.

Important Files Changed

Filename	Overview
dimos/msgs/sensor_msgs/test_PointCloud2.py	Drops tuple unpacking on as_numpy() result in test_lcm_encode_decode, making the point-count assertion trivially true and assert_allclose compare wrong objects.
dimos/mapping/loop_closure/utils/map_rrd.py	New utility script that dumps a recorded SQLite dataset to a Rerun .rrd file with lidar clouds, voxel maps, odometry paths, and camera frames. Logic is clean and consistent with codebase patterns.
dimos/memory2/store/base.py	Adds __contains__ and __iter__ to StreamAccessor and a summary() convenience method to Store.
dimos/msgs/sensor_msgs/PointCloud2.py	Switches height colormap normalization from global min/max to robust 1st–99th percentile clipping.
dimos/utils/data.py	Adds a pull=False fast-fail option to resolve_named_path.
dimos/memory2/observationstore/sqlite.py	Adds a clarifying comment to insert(). No logic changes.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A["map_rrd.py main()"] --> B["resolve_named_path()"]
    B --> C["SqliteStore.open()"]
    C --> D["store.summary()"]
    D --> E{streams present?}
    E --> F["lidar → _log_clouds(world/lidar)"]
    E --> G["fastlio_lidar → _log_clouds(world/fastlio_lidar)"]
    F --> H{--map flag?}
    G --> H
    H -->|yes| I["VoxelMapTransformer → world/lidar_voxels"]
    H -->|yes| J["VoxelMapTransformer → world/fastlio_voxels"]
    E --> K{fastlio_odometry?}
    K -->|yes| L["pose loop → rr.Transform3D (world/fastlio)"]
    L --> M["_log_path (world/fastlio_path)"]
    E --> N{odom?}
    N -->|yes| O["pose loop → rr.Transform3D (world/odom)"]
    O --> P["_log_path (world/odom_path)"]
    E --> Q["color_image (throttled)"]
    Q --> R{image_pose_from?}
    R -->|own| S["img_obs.pose_stamped"]
    R -->|fastlio_odom| T["pose_authority.at(img_obs.ts)"]
    S --> U["rr.Transform3D (world/camera)"]
    T --> U
    U --> V["rr.log(world/camera/image)"]
    V --> W["rr.save → .rrd file"]

Loading

_{Reviews (3): Last reviewed commit: "Merge origin/main into feat/ivan/go2stai..." | Re-trigger Greptile}

[codecov]

❌ 2 Tests Failed:

Tests completed	Failed	Passed	Skipped
2109	2	2107	82

View the top 2 failed test(s) by shortest run time

dimos.msgs.sensor_msgs.test_PointCloud2::test_lcm_encode_decode

Stack Traces | 0.571s run time

@pytest.mark.self_hosted
    def test_lcm_encode_decode() -> None:
        """Test LCM encode/decode preserves pointcloud data."""
        replay = SensorReplay("office_lidar", autocast=pointcloud2_from_webrtc_lidar)
        lidar_msg: PointCloud2 = replay.load_one("lidar_data_021")
    
        binary_msg = lidar_msg.lcm_encode()
        decoded = PointCloud2.lcm_decode(binary_msg)
    
        # 1. Check number of points
        original_points = lidar_msg.as_numpy()
        decoded_points = decoded.as_numpy()
    
        assert len(original_points) == len(decoded_points), (
            f"Point count mismatch: {len(original_points)} vs {len(decoded_points)}"
        )
    
        # 2. Check point coordinates are preserved (within floating point tolerance)
        if len(original_points) > 0:
>           np.testing.assert_allclose(
                original_points,
                decoded_points,
                rtol=1e-6,
                atol=1e-6,
                err_msg="Point coordinates don't match between original and decoded",
            )
E           ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.

binary_msg = b'\xf5\xeb=\xa1\xc2\x851u\x00\x00\x00\x04\x00\x05\xe3\x90\x00\x00\x00\x00h\rX(\x00\x00\x00\x00\x00\x00\x00\x06world\x0...~@33\x83?\x00\x00\x00\x00\xcd\xcc*\xc133\x87@33\x83?\x00\x00\x00\x00\x9a\x99+\xc1ff^@\x9a\x99\x89?\x00\x00\x00\x00\x01'
decoded    = PointCloud(points=24121, frame_id='world', ts=1745705000.0)
decoded_points = (array([[     -6.075,       0.475,      -0.375],
       [     -6.025,       0.475,      -0.375],
       [     -5.975, ...     [    -10.675,       4.225,       1.025],
       [    -10.725,       3.475,       1.075]], shape=(24121, 3)), None)
lidar_msg  = PointCloud(points=24121, frame_id='world', ts=1745705000.0)
original_points = (array([[     -6.075,       0.475,      -0.375],
       [     -6.025,       0.475,      -0.375],
       [     -5.975, ...     [    -10.675,       4.225,       1.025],
       [    -10.725,       3.475,       1.075]], shape=(24121, 3)), None)
replay     = <dimos.memory.timeseries.legacy.LegacyPickleStore object at 0x3a3b02840>

.../msgs/sensor_msgs/test_PointCloud2.py:44: ValueError

dimos.core.coordination.test_worker::test_dedicated_worker_gets_own_process

Stack Traces | 2.68s run time

worker 'gw1' crashed while running '.../core/coordination/test_worker.py::test_dedicated_worker_gets_own_process'

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}

[greptile-apps]

as_numpy() still returns tuple[np.ndarray, np.ndarray | None], so assigning the result directly to original_points and decoded_points silently breaks the test. len(original_points) now returns 2 (the tuple length) rather than the number of points, making the count assertion trivially pass for any pair of clouds. The subsequent np.testing.assert_allclose then receives a 2-element object array [points_array, None] instead of just the point positions — this will either raise a TypeError from numpy's ufunc or produce a meaningless comparison. Note that test_lcm_intensity_round_trip and test_lcm_no_intensity_round_trip in the same file both still use the correct tuple-unpacking pattern.

Suggested change

	# 1. Check number of points
	original_points, _ = lidar_msg.as_numpy()
	decoded_points, _ = decoded.as_numpy()
	original_points = lidar_msg.as_numpy()
	decoded_points = decoded.as_numpy()
	# 1. Check number of points
	original_points, _ = lidar_msg.as_numpy()
	decoded_points, _ = decoded.as_numpy()

[leshy]

Superseded — #2306 includes this recording now.