ThermoSLAM : Stereo Thermal Visual SLAM for Robust Mapping in Degraded Visual Environments

ThermoSLAM is a research project for Stereo Thermal Visual SLAM aimed at robust metric mapping in degraded visual environments (e.g., low light, smoke, fog, and texture-poor scenes).

The pipeline uses stereo thermal imagery to estimate odometry, generate dense depth, synchronize outputs, and build a final map using OctoMap.

---

Project Goal

Build a practical thermal-first SLAM stack that:

• Works when RGB cameras degrade or fail
• Produces metric mapping outputs
• Combines learned odometry and dense stereo depth into a unified map

---

Core Pipeline (ThermoSLAM)

The current ThermoSLAM mapping stack is composed of the following building blocks:

1. Visual Odometry

   • Repository: tartandriver-MAC-VO-ROS2
   • Purpose: produces thermal stereo visual odometry.

2. Dense Stereo Depth

   • Repository: foundation_stereo_ros
   • Purpose: produces dense depth from stereo thermal inputs.

3. Depth/Odometry Synchronization + Transform Matching

   • Code: disparity_odom_sync_postprocess
   • Purpose: synchronizes depth and odometry messages and aligns transforms before mapping.

4. Final Mapping

   • Mapper: Octomap Server 2
   • Purpose: fuses synchronized depth + pose information into final 3D occupancy mapping.

Data Flow Summary

Stereo Thermal Input -> VO (pose) + Dense Depth -> Sync/Postprocess -> Octomap Server 2 -> Final Map

---

Quick Start (ROS 2, Placeholder Commands)

0) Prerequisites

• Ubuntu + ROS 2 installed (currently assumed: ROS 2 Humble)
• A ROS 2 workspace (example: ~/thermoslam_ws)
• Dependencies cloned and available in src/

1) Workspace Setup

git clone --recurse-submodules https://github.com/swastikmhptr/ThermoSLAM.git

cd ThermoSLAM/thermoslam_ws
source /opt/ros/humble/setup.bash
rosdep update
rosdep install --from-paths src --ignore-src -r -y
colcon build --symlink-install
source install/setup.bash

2) Launch Order (ThermoSLAM Pipeline)

Use separate terminals and source your workspace in each:

source /opt/ros/humble/setup.bash
source ~/ThermoSLAM/thermoslam_ws/install/setup.bash

Terminal A - Thermal Stereo Visual Odometry

ros2 launch tartandriver_mac_vo_ros2 vo_thermal_stereo.launch.py

Terminal B - Dense Stereo Depth

ros2 launch foundation_stereo_ros thermal_stereo_depth.launch.py

Terminal C - Depth/Odom Sync + Transform Postprocess

ros2 run disparity_odom_sync_postprocess sync_node \
  --ros-args \
  -p depth_topic:=/foundation/depth/image \
  -p odom_topic:=/tartan/vo/odom \
  -p output_cloud_topic:=/thermoslam/depth_registered \
  -p output_pose_topic:=/thermoslam/pose_synced

Terminal D - Octomap Mapping

ros2 launch octomap_server2 octomap_mapping.launch.py \
  pointcloud_topic:=/thermoslam/depth_registered \
  frame_id:=map

3) Topic Contracts (Placeholder Names)

Expected primary interfaces:

• Thermal stereo images:
  • /thermal/left/image_rect
  • /thermal/right/image_rect
• VO output:
  • /tartan/vo/odom (nav_msgs/msg/Odometry)
• Dense depth output:
  • /foundation/depth/image (sensor_msgs/msg/Image)
• Postprocess synchronized output:
  • /thermoslam/depth_registered (sensor_msgs/msg/PointCloud2)
  • /thermoslam/pose_synced (geometry_msgs/msg/PoseStamped or nav_msgs/msg/Odometry)
• Mapping output:
  • /octomap_full / /octomap_binary (from octomap_server2)

4) Quick Verification Checklist

# Check active topics
ros2 topic list

# Check message rates (examples)
ros2 topic hz /tartan/vo/odom
ros2 topic hz /foundation/depth/image
ros2 topic hz /thermoslam/depth_registered

# Inspect frames
ros2 run tf2_tools view_frames

If mapping is live, you should see:

• stable odometry updates
• synchronized depth/pose stream
• growing OctoMap occupancy output

5) Baseline Quick Run (RGB-D with NVBlox)

# Example placeholder baseline launch
ros2 launch isaac_ros_nvblox nvblox_rgbd.launch.py \
  color_topic:=/camera/color/image_raw \
  depth_topic:=/camera/depth/image_rect_raw \
  camera_info_topic:=/camera/color/camera_info \
  global_frame:=map

Compare ThermoSLAM vs baseline on:

• map completeness
• map consistency
• Average Chamfer Distance
• robustness in degraded visual conditions
• runtime throughput

---

Baseline Setup (RGB-D Mapping)

To establish a baseline, we use:

• NVIDIA Isaac ROS NVBlox
• Input modality: RGB-D
• Goal: compare baseline RGB-D mapping quality/performance against ThermoSLAM thermal-stereo mapping.

This baseline is useful for:

• Quantitative map quality comparisons
• Runtime and robustness comparisons in degraded visual conditions
• Ablation and benchmarking against a known modern mapping stack

---

Dataset Setup: TartanRGBT

Our test dataset is TartanRGBT:

• Hugging Face: theairlabcmu/TartanRGBT
• Size: approximately 79 GB
• Modalities include synchronized RGB, thermal, and depth archives

1) Environment Requirements

Install the Hugging Face client:

pip install huggingface_hub

2) Download + Extraction

The dataset provides a helper extraction script (data_extraction.py) in the dataset repository card instructions.

General usage:

python data_extraction.py <base_dir> [--delete_zips yes|no]

Where:

• <base_dir> is where TartanRGBT_dataset/ will be created
• --delete_zips controls whether zip files are deleted after extraction (no by default)

3) Expected Folder Structure

After extraction, data is organized by day -> trajectory -> modality:

TartanRGBT_dataset/
├── day1/
│   └── <trajectory_name>/
│       ├── thermal_left_rect_8/
│       ├── thermal_right_rect_8/
│       ├── thermal_right_rect_16/
│       ├── thermal_right_left_16/
│       ├── zed_left_rect/
│       ├── zed_right_rect/
│       ├── RGB_aligned_with_thermal/
│       └── stereo_depth/
├── day2/
├── day3/
├── day4/
└── day5/

4) Modalities for This Project

For ThermoSLAM experiments, the key inputs are typically:

• thermal_left_rect_8 / thermal_right_rect_8 (or 16-bit thermal variants)
• stereo_depth (for cross-checking or baseline references)
• RGB_aligned_with_thermal (mainly for baseline and comparison studies)

---

Repository Dependencies

This project relies on external repositories/modules:

• tartandriver-MAC-VO-ROS2 (thermal stereo VO)
• foundation_stereo_ros (dense stereo depth)
• disparity_odom_sync_postprocess (message sync + transform alignment)
• octomap_server2 (final map generation)
• isaac_ros_nvblox (RGB-D baseline mapping)

---