Neural Trajectory POC

This repository is a proof-of-concept demonstrating how to safely integrate non-real-time Deep Learning spatial planners with hard real-time hardware controllers in ROS 2.

It implements a memory-safe, multithreaded architecture that allows a neural network to evaluate environmental constraints and generate spatial waypoints, while a deterministic C++ controller handles dense kinematic interpolation.

Architectural Overview

Integrating AI directly into hardware control loops presents strict timing and memory safety challenges. This project aims to solves this by separating the architecture into three domains:

1. The Spatial Planner

A PyTorch model trained to act as an obstacle-aware waypoint generator. It ingests the robot's current joint state, a target joint state, and the task-space coordinates of an obstacle. It outputs a sparse sequence of safe via-points. The model is exported to ONNX for lightweight inference in C++.

2. The Lock-Free Bridge

Dynamic memory allocation and blocking mutexes are strictly prohibited in the ros2_control hardware loop. To solve this, the architecture utilizes a standard ROS 2 executor thread to run the ONNX inference, and safely passes the resulting waypoints to the hardware thread using realtime_tools::RealtimeBuffer. This lock-free pointer swap ensures the hardware loop never waits for the neural network to finish computing.

3. The Hardware Controller

A custom ros2_control Controller implementation running at 100 Hz. It continuously reads the latest spatial waypoints from the lock-free buffer and evaluates a piece-wise Catmull-Rom Spline. This guarantees that the physical hardware executes a smooth, continuous trajectory with predictable velocities, bridging the gap between the AI's sparse behavioral intent and the motors' strict kinematic limits.

Repository Structure

• planar_manipulator_description: Contains the URDF, simulated hardware interfaces, and launch files for a 2-DOF planar manipulator with a static environmental obstacle.
• neural_trajectory_bridge: The C++ package containing the multithreaded ros2_control node and ONNX Runtime integration.
• waypoint_planner: Python scripts for generating the collision-free geometric dataset (via A* search), training the PyTorch network, and exporting the ONNX model.

Build and Run

1. Build the workspace

colcon build
source install/setup.bash

2. Launch the simulation and controllers

ros2 launch planar_manipulator_description control_manipulator.launch.py

3. Publish a target waypoint Trigger the neural planner by publishing a target joint state and desired duration (in seconds):

ros2 topic pub --once /neural_upscaler/target_waypoints std_msgs/msg/Float64MultiArray "{data: [1.57, 0.0, 5.0]}"

This strips away the debugging history and presents a clean, modular, and highly competent proof-of-concept.