VisionTracker: YOLOv11l + Multi-Object Tracking Benchmark

🎯 Objective:
This project benchmarks three Multi-Object Tracking (MOT) algorithms — SORT, DeepSORT, and OCSORT — using YOLOv11m for object detection on CCTV footage. The goal is to analyze tracker behavior, compare ID stability, and empirically study failure modes in real-world CCTV footage.

---

🚀 Algorithms Implemented

Tracker	Description	Performance Observed
SORT	Simple Online and Realtime Tracking. Uses Kalman filter + Hungarian method.	Fast, but IDs are unstable, frequent ID switches.
DeepSORT	SORT + appearance embedding (ReID) for more stable IDs.	More stable than SORT, still struggles with occlusions.
OCSORT	Advanced online tracker combining motion + appearance heuristics.	Best among the three, smoother trajectories, fewer ID switches, but not perfect.

---

🎥 Dataset

• CCTV footage from MART counter (publicly available on YouTube):
  CCTV Footage
• Duration: ~47 seconds (~1195 frames).
• Objects of interest: primarily people.

Note: Instead of uploading the video, download it manually from YouTube using tools like yt-dlp or youtube-dl and save it locally in the video/ folder.

Example command using yt-dlp:

mkdir -p video
yt-dlp -o "video/CCTV.mp4" https://youtube.com/watch?v=j0xRvqJ67Ug&si=0lUGDR4hOxoJotMb

---

🧠 Model Weights

YOLOv11l pre-trained weights are required. You can download official YOLOv11l weights from the Ultralytics repository or train your own. Save the .pt file in a sota/ folder, e.g., sota/yolov11l.pt. Example: Download pretrained weights:

mkdir -p sota
wget -O sota/yolov11l.pt "https://github.com/ultralytics/ultralytics/releases/download/v0.0.0/yolov11l.pt"

---

🧮 Empirical Tracker Behavior Analysis (Annotation-Free)

Since the dataset lacks annotations, proxy metrics were computed:

Metric	Description	Key Insight
Average / Median Track Length	Duration of continuous object IDs	OCSORT maintained the longest stable tracks
Active Tracks per Frame	Number of ongoing object IDs	DeepSORT and OCSORT balanced detection recall
ID Switch Count	How often IDs are reassigned	SORT had the highest switch rate
Trajectory Smoothness	Bounding box volatility	OCSORT produced the smoothest motion paths

Conclusion: OCSORT consistently outperformed in ID persistence and smoothness across frames.

---

📊 Visualizations

Metric Visualizations

• Track length distribution per tracker
• Active tracks per frame
• Sample trajectories
• Combined plots comparing all three trackers

Tracker-Specific Plots

DeepSORT	OCSORT	SORT

DeepSORT	OCSORT	SORT

DeepSORT	OCSORT	SORT

Combined Tracker Plots

These plots show all three trackers (SORT, DeepSORT, OCSORT) together for easier comparison:

Track Length	Active Tracks	Trajectories

All visualizations are saved automatically and can be used to compare tracker behavior.

---

🖼️ Sample Tracker Outputs

Below are qualitative comparisons across SORT, DeepSORT, and OCSORT at three representative frames in the video sequence
(early, mid, and late — Frames 119, 569, and 1075).

Frame 119

SORT	DeepSORT	OCSORT

Frame 597

SORT	DeepSORT	OCSORT

Frame 1075

SORT	DeepSORT	OCSORT

Observation:

• SORT tends to switch IDs frequently between frames.
• DeepSORT improves stability but drifts slightly under occlusion.
• OCSORT maintains smoother and more persistent tracks over time.

---

---

🎬 Tracked Videos

DeepSORT	OCSORT	SORT

---

⚙️ How to Run

1. Install dependencies:

pip install ultralytics opencv-python tqdm seaborn matplotlib deep_sort_realtime

2. Download the CCTV footage from YouTube or place your video locally.
3. Run the notebook VisionTracker.ipynb to generate tracked videos and plots.
4. All outputs (videos, logs, plots) will be saved automatically.

---

📝 Limitations

1. ID consistency is not perfect, especially for SORT and partially for DeepSORT.
2. OCSORT performs best but still suffers from ID switches in crowded scenes or occlusions.
3. Dataset is short; results may vary with longer or more complex videos.

---

🔮 Future Work / Learning Goals

1. Tune OCSORT hyperparameters (detection threshold, max_age, min_hits) for better ID persistence.
2. Integrate lightweight appearance embeddings for even more stable IDs.
3. Explore longer sequences and more challenging environments.
4. Experiment with additional MOT algorithms (ByteTrack, FairMOT, etc.).

---

📚 Takeaways

1. Tracker choice significantly impacts ID persistence even with identical detections.
2. Appearance embeddings improve stability but do not fully solve occlusion-induced ID switches.
3. OCSORT provides the best trade-off between smooth trajectories and ID consistency in short CCTV sequences.
4. Proxy metrics and qualitative analysis are viable when labeled MOT datasets are unavailable.
5. Learning how to debug, visualize, and analyze tracking performance.
6. Practicing GitHub workflow: sharing, documenting, and versioning work.

---

🧠 Author

Y.R.A.V.R -- Hyderabad, India

• LinkedIn
• GitHub

---

📎 References

• YOLOv11 GitHub
• DeepSORT Paper
• OCSORT GitHub