SARosPerceptionKitti

A ROS package for the Perception (Sensor Processing, Detection, Tracking and Evaluation) of the KITTI Vision Benchmark Suite

Setup

1. Install ROS Kinetic on Ubuntu 16.04
2. Setup ROS Workspace:

mkdir -p ~/catkin_ws/src  
cd ~/catkin_ws/src  
git pull https://github.com/appinho/SARosPerceptionKitti.git  
cd ..  
catkin_make  
source devel/setup.bash  

3. Install Kitti2Bag

pip install kitti2bag

Usage

1. Convert scenario 0060 (from the demo above) into a ROSbag file:

• Download and unzip the synced+rectified data file and its calibration file from the KITTI Raw Dataset
• Merge both files into one ROSbag file

cd ~/kitti_data/
kitti2bag -t 2011_09_26 -r 0060 raw_synced

2. Synchronize the sensor data:

• The script matches the timestamps of the Velodyne point cloud data with the Camara data to perform Sensor Fusion in a synchronized way within the ROS framework

cd ~/catkim_ws/src/ROS_Perception_Kitti_Dataset/pre_processing/
python sync_rosbag.py raw_synced.bag

3. Store preprocessed semantic segmentated images:

• The Camera data is preprocessed within a Deep Neural Network to create semantic segmentated images. With this step a "real-time" performance on any device (CPU usage) can be guaranteed

mkdir ~/kitti_data/0060/segmented_semantic_images/
cd ~/kitti_data/0060/segmented_semantic_images/

• For scenario 0060 you can download my results and store them within the directory

• For any other scenario follow this steps: Well pre-trained network with an IOU of 73% can be found here: Finetuned Google's DeepLab on KITTI Dataset

4. Final folder structure

    ~                                        # Home directory
    ├── catkin_ws                            # Catkin workspace
    │   ├── src                              # Clone repo in here
    │       └── ROS_Perception_Kitti_Dataset # Repo
    ├── kitti_data                           # Data folder
    │   ├── 0001                             # Scenario 0001
    │   ├── ...                              # Any other scenario
    │   ├── 0060                             # Demo scenario 0060
    │   │   ├── segmented_semantic_images    # Folder for semantic images (Copy download in here)
    │   │   │   ├── 0000000000.png           # Semantic image from first time frame
    │   │   │   ├── 0000000001.png           # Semantic image from second time frame
    │   │   │   └── ...  
    │   │   └── synchronized_data.bag        # Synchronized ROSbag file
    │   ├── ...

5. Run the ROS Package:

• Launch one of the following ROS nodes together with the scenario identifier and wait until RViz is fully loaded:

roslaunch sensor_processing sensor.launch scenario:=0060
roslaunch detection detection.launch scenario:=0060
roslaunch tracking tracking.launch scenario:=0060
roslaunch evaluation evaluation.launch scenario:=0060

• Play back the synchronized ROSbag file (here at 25% speed):

cd ~/kitti_data/0060/
rosbag play -r 0.25 synchronized_data.bag

Troubleshooting

• SEMANTIC IMAGES WARNING: Go to sensor.cpp line 542 in sensor_processing_lib and hardcode your personal home directory! (see full discussion here)

• Make sure to close RVIz and restart the ROS launch command if you want to execute the scenario again. Otherwise it seems like the data isn't moving anymore (see here)

• Make sure the scenario is encoded as 4 digit number, like above 0060

• Make sure the images are encoded as 10 digit numbers starting from 0000000000.png

• Make sure the resulting semantic segmentated images have the color encoding of the Cityscape Dataset

Discussion

Evaluation results for 7 Scenarios 0011,0013,0014,0018,0056,0059,0060

Class	MOTP	MODP
Car	0.715273	0.785403
Pedestrian	0.581809	0.988038

Areas for Improvements

• Improving the Object Detection so that the object's shape, especially for cars, is incorporated and that false classification within the semantic segmentation can be tolerated
• Applying the VoxelNet

Contact

Send me an email simonappel62@gmail.com if you have any questions, wishes or ideas to find an even better solution! Happy to collaborate :)