Skip to content

Develop VLM detection and training - #798

Draft
EHxuban11 wants to merge 22 commits into
devfrom
vlm-detection-roadmap
Draft

Develop VLM detection and training#798
EHxuban11 wants to merge 22 commits into
devfrom
vlm-detection-roadmap

Conversation

@EHxuban11

@EHxuban11 EHxuban11 commented Aug 15, 2026

Copy link
Copy Markdown
Contributor
  • Adds VLM detection training, native COCO support, checkpoint-bound confidence validation on holdout100, and immutable reviewed VLM Hub artifacts.
  • Adds strict Qwen3-VL LoRA identity/publication, offline preflight, dataset attestation, immutable remote routing, and a canonical exact-zero two-run repeatability receipt.
  • Receipt publication binds both report/envelope pairs, preserves the strict comparator output, rejects path-bearing diagnostics, and revalidates source plus private snapshot bytes before create-only publication.
  • Verified with 941 broad offline VLM/API/training/Hub/routing tests and 538 post-integration benchmark/dataset/validator/report/artifact tests; Ruff, formatting, diff checks, and Greptile passed.
  • RTX 5070 Ti smoke: Qwen3-VL 2B LoRA train/reload/predict passed; no long benchmark was run.
  • Not live-tested: Hugging Face repository creation/upload and full Qwen 4B weight loading. North Micro Vision remains release-gated until official transformers>=5.16.0 exists.

Code provenance

  • Original code written for this PR except the permissively licensed sources listed below and recorded in THIRD_PARTY_NOTICES.txt.
  • Shared COCO bounding-box clipping helper: Megvii-BaseDetection/YOLOX commit 6d30efe8f9f277e8788fb2d9b17d08238a5ea292, yolox/data/datasets/coco.py (Apache-2.0).
  • The Qwen3-VL LoRA/frozen-vision strategy references QwenLM/Qwen3-VL commit 96588727e44c78b25ba03ea03b8e12f7e64fd0da, paths qwen-vl-finetune/README.md, qwen-vl-finetune/qwenvl/train/argument.py, and qwen-vl-finetune/qwenvl/train/train_qwen.py (Apache-2.0). LibreYOLO uses its own projection scope, hyperparameters, and detection pipeline.
  • No GPL, AGPL, LGPL, non-commercial, proprietary, or unknown-license code is introduced.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant