A computer vision pipeline for detecting and tracking multiple entities across a scene, mapping their positions to real-world coordinates, and extracting behavioural analytics. Built as a general framework with domain-specific extensions.
The core pipeline handles what is common to any tracking problem: validating video input, computing a homography to translate pixel positions into real-world coordinates (i.e., meters), running an object detector, and maintaining entity identities across occlusions and camera changes via an explicit state machine. This layer is domain-agnostic; it does not assume what is being tracked.
Domain extensions plug into this layer and add the specifics: what model to use, how entities are re-identified, what events to detect, and what the output analytics look like. The two domains currently in scope are handball player tracking and welfare monitoring for a free-roaming laying hen flock.
The immediate application is a monitoring system for a large free-roaming flock (~2000 hens). The system is designed in two layers. Layer 1 delivers useful outputs without needing to individually identify birds. Or more concrete, it treats identity as a location problem, not an identity problem. Layer 2 adds per-individual tracking once a reliable re-identification mechanism is in place. This separation de-risks the project.
Layer 1 targets:
- Death detection: flag a stationary bird's location within 30 minutes
- Sickness detection: flag birds showing sustained isolation or depressed local activity
- Event recognition: feeding episodes, egg-laying, fights (detected by proximity to fixed locations and movement signatures)
- Weight estimation: inferred from overhead silhouette area and body dimensions
Layer 2 targets (deferred until Layer 1 is validated):
- Per-individual feeding history, weight trajectory, and longitudinal health records
- These require stable individual identity across time and cameras — the hardest sub-problem in the system
Re-identification at 800-bird scale seems to be an open research problem for uniform-colour breeds without external markers. One idea is to use a low-cost marking scheme (likely small QR patches on each bird's back, readable from the overhead cameras directly) rather than attempt vision-only Re-ID.
Detection and tracking. YOLOv8 instance segmentation fine-tuned on chicken-specific data, combined with ByteTrack for multi-object tracking. Instance segmentation (rather than bounding-box detection) is used throughout because the silhouette mask is needed for weight estimation and body-axis orientation.
Coordinate system. Each camera's image plane is mapped to a shared real-world ground plane via homography. All downstream analytics operate in metres, not pixels. For the mostly overhead camera configuration used here, the image plane approximates the ground plane and the homography is well-conditioned.
Multi-camera. The shed requires multiple overhead cameras. Tracks from individual cameras are fused on the shared ground plane by proximity matching. Cross-camera identity handoff uses spatial-temporal proximity — no appearance embedding needed for Layer 1.
Behavioural analytics. Death and sickness detection are treated as spatial anomaly problems (sustained immobility; spatial isolation from the flock). Event detection (feeding, egg-laying) uses fixed-zone proximity triggers against known feeder and nest-box locations. Weight regression maps silhouette features to grams using a model trained on manually weighed reference birds.
Development without own hardware. Four public datasets cover the development phase before cameras are installed in a real shed: MVBroTrack (four synchronised overhead cameras, annotated detection and tracking across the broiler life cycle), PIO (large-scale detection data from commercial sheds), Chicks4FreeID (individual Re-ID benchmark), and Chicken-Behavior-Analysis (inactivity and huddling labels). The full detection, tracking, and Layer 1 analytics stack can be validated on these datasets before a single camera is installed.
motion-tracking/
├── config/ # all tunable parameters — single source of truth
├── src/
│ ├── ingest/ # dataset parsers + canonical data schema
│ ├── detection/ # YOLO wrapper + fine-tuning utilities
│ ├── tracking/ # ByteTrack integration + state machine
│ ├── homography/ # H estimation + world coordinate mapping
│ ├── analytics/ # death, sickness, feeding, egg-laying, fighting
│ ├── fusion/ # multi-camera calibration + track fusion
│ ├── weight/ # silhouette-based weight estimation
│ └── reid/ # Layer 2 — individual Re-ID (deferred)
├── data/
│ ├── raw/ # downloaded datasets — gitignored
│ └── processed/ # normalised splits — gitignored
├── tests/
│ └── kill_checks/ # automated stage-gate validation scripts
├── notebooks/ # exploratory work
└── scripts/ # dataset download + preparation utilities
The virtual environment is configured:
source .venv/bin/activate # Windows: .venv\Scripts\activate
pip install ultralytics supervision opencv-python numpy scipy shapely lap pyyaml scikit-learnVerify GPU access before anything else:
python -c "import torch; print(torch.cuda.is_available())"If this returns False on a GPU machine, fix the CUDA/PyTorch installation first because the detection and tracking stack requires it.
The chicken monitoring work sits within the Precision Livestock Farming (PLF) field. Key references:
- MVBroTrack — multi-camera broiler tracking dataset
- Chicks4FreeID — individual Re-ID benchmark
- PIO — large-scale broiler detection dataset
- Biermann et al. (2021) — EIGD handball dataset, ground-truth backbone for the handball extension