Aerial Guardian — Drone Person Detection & Tracking

A lightweight, drone-specific pipeline for detecting and tracking persons from a moving UAV platform. Built on fine-tuned YOLOv8n, Adaptive SAHI tiled inference, ByteTrack, and homography-based ego-motion compensation.

Key constraint met: Total model weight ~6 MB (well under 300 MB limit). No ReID network required.

---

Pipeline Overview

Frame sequence
     │
     ▼
Ego-motion homography (ORB + RANSAC)   ← CPU, runs every frame
     │
     ▼
Detection — every N frames: full SAHI   otherwise: adaptive tiled inference
     │
     ▼
Homography warp on detections           ← compensates camera motion before tracker
     │
     ▼
ByteTrack (direct detection feed)       ← no internal re-detection
     │
     ▼
Annotated video (boxes + IDs + tails)

---

Results

Detection (YOLOv8n fine-tuned on VisDrone persons)

Metric	Value
mAP@0.5	86.5%
mAP@0.5:0.95	49.9%
Precision	88.1%
Recall	77.7%
Model size	~6 MB
Training epochs	100

Trained and evaluated on the VisDrone2019-MOT-val set (the only provided split). Train and val curves share the same image pool — results reflect in-distribution performance.

Training Curves

All losses (train and val) decrease steadily across 100 epochs with no divergence. mAP@0.5 rises from 53% at epoch 1 to 86.5% at epoch 95 — no overfitting observed.

Validation Predictions

Persons detected at varying scales and densities from aerial perspective across multiple scene types.

Training Samples

Mosaic augmentation combines four aerial scenes per batch, exposing the model to many small persons simultaneously.

---

Pipeline FPS

Hardware: NVIDIA RTX 4060 | Sequence: uav0000137_00458_v

Configuration	FPS	Avg tiles/frame	New-entrant latency
Standard SAHI, interval=1	3.31	28	1 frame
Adaptive SAHI, interval=1	5.21	14	1 frame
Adaptive SAHI, interval=2	8.83	10	≤2 frames
Adaptive SAHI, interval=3	10.58	10	≤3 frames
Adaptive SAHI, interval=4	11.77	10	≤4 frames
Adaptive SAHI, interval=7	13.04	10	≤7 frames

Recommended: --det-interval 3 — 3.2× faster than standard SAHI baseline with acceptable detection latency for new persons entering the scene.

Projected Jetson Orin Nano performance (not measured on device — estimated from published TensorRT benchmarks):

• TensorRT FP16, no SAHI: ~25–35 FPS
• TensorRT FP16, adaptive SAHI, interval=3: ~8–12 FPS

---

Repository Structure

aerial-guardian/
├── adaptive_sahi.py          # Adaptive tile selection — centres tiles on prior detections
├── detect_track.py           # Main pipeline: SAHI → homography → ByteTrack → video
├── homography_stabilizer.py  # ORB-based ego-motion estimation and box warping
├── train.py                  # Fine-tune YOLOv8n on VisDrone person class
├── visdrone_to_yolo.py       # Convert VisDrone MOT annotations → YOLO format
├── weights/
│   └── best.pt               # Fine-tuned YOLOv8n weights (~6 MB)
├── result/
│   ├── results.png
│   ├── val_batch1_pred.jpg
│   └── train_batch1.jpg
├── requirements.txt
├── SUMMARY.md
└── README.md

---

Setup

1. Clone and create environment

git clone <repo_url>
cd aerial-guardian
python3 -m venv .venv
source .venv/bin/activate   

2. Install dependencies

pip install -r requirements.txt
pip install lapx>=0.5.2

3. Download the dataset

Download VisDrone2019-MOT-val and place it at:

aerial-guardian/
└── VisDrone2019-MOT-val/
    ├── annotations/
    └── sequences/

---

Usage

Step 1 — Convert dataset to YOLO format

python visdrone_to_yolo.py ./VisDrone2019-MOT-val ./visdrone_yolo

Filters VisDrone categories 1 (pedestrian) and 2 (person) → YOLO class 0. Produces normalized cx cy w h labels per frame.

Step 2 — Fine-tune YOLOv8n

python train.py \
  --data ./visdrone_yolo/dataset.yaml \
  --epochs 100 \
  --batch 8 \
  --img 640 \
  --freeze 5 \
  --device 0

Best weights saved to runs/train-100epochs/visdrone_person/weights/best.pt. Copy to weights/best.pt before running inference.

Step 3 — Run detection and tracking on a single sequence

python detect_track.py \
  --weights weights/best.pt \
  --source ./VisDrone2019-MOT-val/sequences/uav0000086_00000_v \
  --output output_uav0000086.mp4 \
  --conf 0.20 \
  --det-interval 3 \
  --device 0

Step 4 — Run on all sequences

for seq in ./VisDrone2019-MOT-val/sequences/*/; do
  name=$(basename "$seq")
  python detect_track.py \
    --weights weights/best.pt \
    --source "$seq" \
    --output "output_${name}.mp4" \
    --conf 0.20 \
    --det-interval 3 \
    --device 0
done

---

Output Video

Google Drive — processed sequences with bounding boxes, IDs, and trajectory tails