From d57b0db24b6e33c2fbc394358da1682c4d2e01c5 Mon Sep 17 00:00:00 2001 From: George Pearse Date: Fri, 2 Jan 2026 19:41:03 +0000 Subject: [PATCH] docs: add DoRA to roadmap MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add DoRA (Weight-Decomposed Low-Rank Adaptation) to the Other Planned Integrations section. DoRA improves upon LoRA by decomposing weights into magnitude and direction components, offering better performance especially at lower ranks with no inference overhead. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 --- docs/roadmap.md | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/docs/roadmap.md b/docs/roadmap.md index a66a4278..a6d58518 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -280,6 +280,22 @@ GPU-accelerated preprocessing for NVIDIA hardware: - Pipeline-parallel execution - Best for high GPU:CPU ratio systems +### DoRA (Weight-Decomposed Low-Rank Adaptation) + +**Status**: Under evaluation + +[DoRA](https://github.com/NVlabs/DoRA) is a parameter-efficient fine-tuning method that improves upon LoRA by decomposing pre-trained weights into magnitude and direction components: +- **Superior to LoRA** especially at lower ranks, allowing reduced memory consumption +- **No inference overhead** - zero additional computational cost during inference +- **Enhanced training stability** through weight decomposition approach +- **ICML 2024 Oral** (top 1.5% acceptance rate) +- **HuggingFace PEFT integration** - supports Linear, Conv1d, Conv2d layers and quantized models + +Potential applications for visdet: +- Efficient fine-tuning of vision transformer backbones (Swin, ViT) +- Adaptation of detection heads for domain-specific tasks +- Low-resource model customization with minimal trainable parameters + --- ## Timeline Summary