Skip to content
BranchesTags

Repository files navigation

MultiCamZMQ

C++17 Python License CMake

MultiCamZMQ is a high-performance, low-latency multi-camera video streaming library built on top of ZeroMQ. It enables seamless real-time video transmission from multiple cameras across processes and machines with minimal overhead.

Key Features

  • High Performance: Zero-copy operations, CPU core pinning, and optimized frame serialization
  • Multiple Transport Protocols: TCP, IPC, inproc via ZeroMQ
  • Dual API Design: Simple ImageServer/Client API and advanced FramePublisher/Subscriber API
  • Compression Options: RAW (uncompressed) and JPEG compression modes
  • Sub-millisecond Latency: Timestamp support for precise latency measurements
  • Flexible Input Sources: Camera devices, video files, or manual frame injection
  • Python Bindings: Full-featured Python API via pybind11
  • Thread-Safe: Built-in thread management and synchronization
  • Pub-Sub Pattern: Topic-based subscription with ZMQ PUB-SUB pattern
  • CPU Affinity: Pin threads to specific CPU cores for optimal performance

Architecture

┌─────────────────────────────────────────────────────────────────┐
│                        MultiCamZMQ                              │
├─────────────────────────────────────────────────────────────────┤
│                                                                 │
│   ┌─────────────────┐              ┌─────────────────┐        │
│   │ FramePublisher  │─────ZMQ─────→│ FrameSubscriber │        │
│   │                 │  (PUB-SUB)    │                 │        │
│   │ • Camera/Video  │              │ • Topic filter   │        │
│   │ • CPU pinning   │              │ • CPU pinning    │        │
│   │ • RAW/JPEG      │              │ • RAW/JPEG       │        │
│   │ • Timestamps    │              │ • Callbacks      │        │
│   └─────────────────┘              └─────────────────┘        │
│                                                                 │
│   ┌─────────────────┐              ┌─────────────────┐        │
│   │  ImageServer    │─────ZMQ─────→│  ImageClient    │        │
│   │                 │  (REQ-REP)    │                 │        │
│   │ • Simple API    │              │ • Auto-connect   │        │
│   │ • Video stream  │              │ • Multi-camera   │        │
│   └─────────────────┘              └─────────────────┘        │
│                                                                 │
├─────────────────────────────────────────────────────────────────┤
│              OpenCV │ ZeroMQ │ nlohmann/json                   │
└─────────────────────────────────────────────────────────────────┘

Use Cases

  • Multi-Camera Surveillance: Stream from multiple IP or USB cameras simultaneously
  • Computer Vision Pipelines: Decouple camera capture from processing
  • Distributed Systems: Stream video between machines over network
  • Robotics: Multi-sensor visual data acquisition
  • Video Production: Real-time multi-camera monitoring
  • Machine Learning: Efficient data pipeline for video-based ML models

Prerequisites

System Dependencies

Ubuntu/Debian

sudo apt-get update
sudo apt-get install -y \
    build-essential \
    cmake \
    git \
    libopencv-dev \
    libzmq3-dev \
    nlohmann-json3-dev \
    pkg-config

macOS

brew install cmake opencv zeromq nlohmann-json pkg-config

Arch Linux

sudo pacman -S cmake opencv zeromq nlohmann-json

Python Dependencies (for Python bindings)

pip install numpy opencv-python

Building the Project

1. Clone the Repository

git clone https://github.com/yourusername/MultiCamZMQ.git
cd MultiCamZMQ

2. Build ZeroMQ Libraries (if not using system packages)

The project includes ZeroMQ as a submodule. If you want to use the included version:

# Build libzmq
cd ZMQ/libzmq
mkdir -p build && cd build
cmake .. -DCMAKE_BUILD_TYPE=Release \
         -DENABLE_DRAFTS=OFF \
         -DBUILD_TESTS=OFF \
         -DWITH_DOCS=OFF
make -j$(nproc)
sudo make install
cd ../../..

# Build cppzmq
cd ZMQ/cppzmq
mkdir -p build && cd build
cmake .. -DCMAKE_BUILD_TYPE=Release \
         -DCPPZMQ_BUILD_TESTS=OFF
make -j$(nproc)
sudo make install
cd ../../..

3. Build C++ Library

cd MultiCamStreamer/cpp
mkdir -p build && cd build

cmake .. -DCMAKE_BUILD_TYPE=Release
make -j$(nproc)

# Optional: Install system-wide
sudo make install

This builds:

  • libMultiCamStreamer.a - Static library
  • sender - Example publisher using ImageServer
  • receiver - Example subscriber using ImageClient
  • frame_pub - Example publisher using FramePublisher
  • frame_sub - Example subscriber using FrameSubscriber

4. Build Python Bindings

# Install pybind11
pip install pybind11[global]

# Build Python module
cd ../../python
mkdir -p build && cd build

cmake .. -DCMAKE_BUILD_TYPE=Release
make -j$(nproc)

# The .so file is automatically copied to multicam_streamer/ package
cd ..

Quick Start

C++ Example

Publisher (Camera Stream)

#include "multicam_streamer/frame_publisher.hpp"
#include <thread>

int main() {
    // Create publisher for camera 0
    FramePublisher publisher(
        "tcp://*:5555",                      // Endpoint
        0,                                    // Camera ID
        0,                                    // CPU core
        FramePublisher::Mode::RAW,           // RAW or JPEG
        640, 480,                            // Resolution
        80,                                   // JPEG quality
        30,                                   // FPS
        true,                                 // Include timestamp
        FramePublisher::SourceType::CAMERA   // Camera or VIDEO_FILE
    );
    
    publisher.run();  // Blocking call
    return 0;
}

Subscriber (Receive Frames)

#include "multicam_streamer/frame_subscriber.hpp"
#include <opencv2/opencv.hpp>

void on_frame(int camera_id, const std::string& topic, 
              const cv::Mat& frame, int64_t timestamp_ns) {
    cv::imshow("Camera " + std::to_string(camera_id), frame);
    cv::waitKey(1);
}

int main() {
    FrameSubscriber subscriber(
        0,                               // Camera ID
        "tcp://localhost:5555",          // Endpoint
        "cam0",                          // Topic to subscribe
        1,                               // CPU core
        FrameSubscriber::Mode::RAW,      // RAW or JPEG
        on_frame,                        // Callback
        true                             // Expects timestamp
    );
    
    subscriber.start();  // Non-blocking
    std::this_thread::sleep_for(std::chrono::hours(1));
    subscriber.stop();
    return 0;
}

Python Example

Publisher (Video File Stream)

from multicam_streamer import FramePublisher, PublisherMode, PublisherSourceType
import time

publisher = FramePublisher(
    endpoint="tcp://*:5555",
    camera_id=0,
    cpu_core=0,
    mode=PublisherMode.RAW,
    width=640,
    height=480,
    fps=30,
    source_type=PublisherSourceType.VIDEO_FILE,
    video_file_path="video.mp4"
)

publisher.run()  # Blocking - streams video in loop

Subscriber (Receive and Display)

from multicam_streamer import FrameSubscriber, SubscriberMode
import cv2
import time

def on_frame(camera_id, topic, frame, timestamp_ns):
    cv2.imshow(f"Camera {camera_id}", frame)
    cv2.waitKey(1)

subscriber = FrameSubscriber(
    camera_id=0,
    endpoint="tcp://localhost:5555",
    topic="cam0",
    cpu_core=1,
    mode=SubscriberMode.RAW,
    callback=on_frame
)

subscriber.start()  # Non-blocking

try:
    while True:
        time.sleep(1)
except KeyboardInterrupt:
    subscriber.stop()
    cv2.destroyAllWindows()

Advanced Usage

Multiple Camera Streams

Python - Multiple Publishers

import threading
from multicam_streamer import FramePublisher, PublisherMode, PublisherSourceType

def run_publisher(camera_id, port):
    publisher = FramePublisher(
        endpoint=f"tcp://*:{port}",
        camera_id=camera_id,
        cpu_core=camera_id % 4,  # Distribute across cores
        mode=PublisherMode.RAW,
        width=640,
        height=480,
        fps=30,
        source_type=PublisherSourceType.CAMERA
    )
    publisher.run()

# Start multiple publishers in separate threads
threads = []
for i, port in enumerate([5555, 5556, 5557]):
    t = threading.Thread(target=run_publisher, args=(i, port), daemon=True)
    t.start()
    threads.append(t)

# Keep main thread alive
for t in threads:
    t.join()

Python - Multiple Subscribers

from multicam_streamer import FrameSubscriber, SubscriberMode
import cv2

subscribers = []
configs = [
    {"camera_id": 0, "endpoint": "tcp://localhost:5555", "cpu_core": 0},
    {"camera_id": 1, "endpoint": "tcp://localhost:5556", "cpu_core": 1},
    {"camera_id": 2, "endpoint": "tcp://localhost:5557", "cpu_core": 2},
]

def on_frame(camera_id, topic, frame, timestamp_ns):
    cv2.imshow(f"Camera {camera_id}", frame)
    cv2.waitKey(1)

# Create and start all subscribers
for config in configs:
    subscriber = FrameSubscriber(
        camera_id=config["camera_id"],
        endpoint=config["endpoint"],
        topic=f"cam{config['camera_id']}",
        cpu_core=config["cpu_core"],
        mode=SubscriberMode.RAW,
        callback=on_frame
    )
    subscriber.start()
    subscribers.append(subscriber)

# Main loop
try:
    while True:
        time.sleep(0.1)
except KeyboardInterrupt:
    for sub in subscribers:
        sub.stop()

Network Streaming

Stream between different machines:

Machine 1 (Publisher):

publisher = FramePublisher(
    endpoint="tcp://*:5555",  # Listen on all interfaces
    camera_id=0,
    cpu_core=0,
    mode=PublisherMode.JPEG,  # Use JPEG for network efficiency
    width=1280,
    height=720,
    fps=30,
    jpeg_quality=85,
    source_type=PublisherSourceType.CAMERA
)
publisher.run()

Machine 2 (Subscriber):

subscriber = FrameSubscriber(
    camera_id=0,
    endpoint="tcp://192.168.1.100:5555",  # Publisher's IP
    topic="cam0",
    cpu_core=0,
    mode=SubscriberMode.JPEG,
    callback=on_frame
)
subscriber.start()

Compression Modes

RAW Mode (No compression):

  • Lowest latency
  • Best quality
  • Higher bandwidth usage
  • Use case: Local IPC, low-latency requirements

JPEG Mode (Lossy compression):

  • Lower bandwidth usage
  • Network-friendly
  • Slight quality loss
  • CPU overhead for encoding/decoding
  • Use case: Network streaming, bandwidth-constrained scenarios

Performance Tuning

CPU Core Pinning

Pin publisher and subscriber threads to different CPU cores for optimal performance:

publisher = FramePublisher(
    endpoint="tcp://*:5555",
    camera_id=0,
    cpu_core=0,  # Pin to core 0
    mode=PublisherMode.RAW,
    # ... other params
)

subscriber = FrameSubscriber(
    camera_id=0,
    endpoint="tcp://localhost:5555",
    topic="cam0",
    cpu_core=2,  # Pin to core 2 (avoid same core)
    mode=SubscriberMode.RAW,
    callback=on_frame
)

Latency Measurement

def on_frame(camera_id, topic, frame, timestamp_ns):
    import time
    current_time_ns = int(time.time() * 1e9)
    latency_ms = (current_time_ns - timestamp_ns) / 1e6
    print(f"Latency: {latency_ms:.2f} ms")

Running Examples

C++ Examples

cd MultiCamStreamer/cpp/build

# Terminal 1: Start publisher
./sender

# Terminal 2: Start subscriber
./receiver

# Or use frame-based examples
./frame_pub
./frame_sub

Python Examples

cd MultiCamStreamer/python/examples

# Terminal 1: Start publisher
python simple_publisher.py

# Terminal 2: Start subscriber
python simple_subscriber.py

# Run performance tests
python performance_test.py

Performance Benchmarks

Typical performance metrics on modern hardware (Intel i7, 16GB RAM):

Resolution Mode FPS Latency Bandwidth CPU Usage
640x480 RAW 60 <1ms ~55 MB/s Low
640x480 JPEG 60 ~2ms ~5 MB/s Medium
1280x720 RAW 30 <1ms ~66 MB/s Low
1280x720 JPEG 30 ~3ms ~8 MB/s Medium
1920x1080 RAW 30 <2ms ~149 MB/s Medium
1920x1080 JPEG 30 ~4ms ~12 MB/s Medium

Note: Performance varies based on hardware, network conditions, and system load.

API Reference

C++ API

FramePublisher

class FramePublisher {
public:
    enum class Mode { JPEG, RAW };
    enum class SourceType { CAMERA, VIDEO_FILE };
    
    FramePublisher(
        const std::string& endpoint,
        int camera_id,
        int cpu_core,
        Mode mode,
        int width = 640,
        int height = 480,
        int jpeg_quality = 80,
        int fps = 30,
        bool include_timestamp = true,
        SourceType source_type = SourceType::CAMERA,
        const std::string& video_file_path = ""
    );
    
    void run();  // Blocking
};

FrameSubscriber

class FrameSubscriber {
public:
    enum class Mode { JPEG, RAW };
    using FrameCallback = std::function<void(int camera_id, 
                                             const std::string& topic, 
                                             const cv::Mat& frame, 
                                             int64_t timestamp_ns)>;
    
    FrameSubscriber(
        int camera_id,
        const std::string& endpoint,
        const std::string& topic,
        int cpu_core,
        Mode mode,
        FrameCallback cb,
        bool includes_timestamp = true
    );
    
    void start();  // Non-blocking
    void stop();
};

Python API

All C++ classes and enums are exposed to Python with the same interface:

from multicam_streamer import (
    FramePublisher,
    FrameSubscriber,
    ImageServer,
    ImageClient,
    PublisherMode,        # JPEG, RAW
    SubscriberMode,       # JPEG, RAW
    PublisherSourceType,  # CAMERA, VIDEO_FILE
    ImageServerSourceType
)

Troubleshooting

Common Issues

1. CMake can't find ZeroMQ

# Set ZeroMQ paths manually
cmake .. -DZeroMQ_DIR=/usr/local/lib/cmake/ZeroMQ

2. OpenCV not found

# Ubuntu/Debian
sudo apt-get install libopencv-dev

# Or specify OpenCV path
cmake .. -DOpenCV_DIR=/usr/local/lib/cmake/opencv4

3. Python module import error

# Add parent directory to Python path
import sys
sys.path.append('/path/to/MultiCamStreamer/python')

4. High latency

  • Use RAW mode instead of JPEG
  • Enable CPU core pinning
  • Reduce resolution or FPS
  • Use IPC instead of TCP for local communication

5. Dropped frames

  • Increase ZMQ buffer size
  • Check network bandwidth
  • Reduce number of simultaneous streams
  • Use faster codec (RAW)

Project Structure

MultiCamZMQ/
├── MultiCamStreamer/
│   ├── cpp/
│   │   ├── include/
│   │   │   └── multicam_streamer/
│   │   │       ├── frame_publisher.hpp
│   │   │       ├── frame_subscriber.hpp
│   │   │       ├── image_server.hpp
│   │   │       └── image_client.hpp
│   │   ├── src/
│   │   │   ├── frame_publisher.cpp
│   │   │   ├── frame_subscriber.cpp
│   │   │   ├── image_server.cpp
│   │   │   └── image_client.cpp
│   │   ├── examples/
│   │   │   ├── sender.cpp
│   │   │   ├── receiver.cpp
│   │   │   ├── frame_pub.cpp
│   │   │   └── frame_sub.cpp
│   │   └── CMakeLists.txt
│   ├── python/
│   │   ├── multicam_streamer/
│   │   │   ├── __init__.py
│   │   │   └── multicam_streamer.pyi  # Type hints
│   │   ├── examples/
│   │   │   ├── simple_publisher.py
│   │   │   ├── simple_subscriber.py
│   │   │   ├── performance_test.py
│   │   │   └── video_file_publisher.py
│   │   ├── src/
│   │   │   └── multicam_streamer_bindings.cpp
│   │   └── CMakeLists.txt
│   └── sample_videos/
│       └── sample_video_1.mp4
├── ZMQ/
│   ├── libzmq/      # ZeroMQ core library
│   └── cppzmq/      # C++ bindings for ZeroMQ
├── extern/
│   └── pybind11/    # Python binding generator
└── README.md

Contributing

Contributions are welcome! Please feel free to submit pull requests or open issues for bugs and feature requests.

Development Setup

git clone https://github.com/yourusername/MultiCamZMQ.git
cd MultiCamZMQ
git submodule update --init --recursive

# Create a feature branch
git checkout -b feature/your-feature-name

# Make changes and test

# Submit PR

License

This project is licensed under the MIT License - see the LICENSE file for details.

Acknowledgments

  • ZeroMQ - High-performance asynchronous messaging library
  • OpenCV - Computer vision library
  • pybind11 - Seamless C++/Python bindings
  • nlohmann/json - JSON for Modern C++

Contact

For questions or support, please open an issue on GitHub.

Made with ❤️ for the computer vision and robotics community

About

No description, website, or topics provided.

Resources

Readme
MIT license
Activity

Stars

1 star

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages