This project implements a distributed video processing system using a master–worker model. You control everything from a simple UI, while worker nodes handle video chunks in parallel.
- 🖥️ Streamlit for the master UI
- 🌐 Flask for worker communication
- 🎞️ FFmpeg for encoding and merging
- ✂️ MP4Box for video splitting
- 🧩 Multiple worker nodes for parallel processing
The master node performs the following steps automatically:
- 📥 Accepts a large input MP4 file
- ✂️ Splits the video into time-based chunks
- 📤 Assigns chunks to available worker nodes
- ⏳ Waits for all workers to finish processing
- 🔗 Merges processed chunks into a final MP4
- 🧹 Cleans up all temporary chunk and output folders
Install using pip:
- flask
- werkzeug
- streamlit
- streamlit-autorefresh
- pandas
- requests
Only the following are required:
- requests
- ffmpeg
Used for splitting videos into chunks.
- Command used:
MP4Box -split
<seconds> - Download via official GPAC site or YouTube guides
- Ensure MP4Box is added to system PATH
Used for encoding and merging video chunks.
Install on Windows:
winget install ffmpeg
Verify both ffmpeg and MP4Box are accessible from the terminal.
Start the master controller UI:
streamlit run master.py
- Master node listens on port 5000
- Ensure inbound TCP traffic is allowed on this port
- Add a new inbound rule
- Protocol: TCP
- Port: 5000
- Action: Allow
This project demonstrates:
- Distributed computing fundamentals
- Parallel video encoding
- Master–worker architecture
- Practical use of FFmpeg in Python systems
Ideal for academic experiments, demos, and learning distributed systems through multimedia processing.