arXiv:2608.22368cs.CVcs.LG2026-08

无需标签,87分钟将ViT检测模型转为线性注意力,速度提升62%。

DiD It in 87 Minutes: A Label-Free Softmax-to-Linear Adaptation of Vision Transformers for Object Detection

论文配图:DiD It in 87 Minutes: A Label-Free Softmax-to-Linear Adaptation of Vision Transformers for Object Detection
图 1 · 摘自论文原文
  • 通过对齐检测器接口特征,实现无标签转换
  • DOTA-v1.5上性能超基线,媲美全训练线性模型
  • 适合想高效复用已有检测模型的研究者

线性注意力因计算成本低,适用于高分辨率目标检测,但将已训练的Softmax注意力ViT骨干网络直接替换为线性注意力会导致性能严重下降。通用的无标签蒸馏虽在分类任务有效,但在检测任务常失败。核心挑战在于‘检测器接口保持’:转换后的骨干需输出下游检测器期望的特征张量,而非仅模仿内部Softmax隐藏状态。为此,提出检测器接口蒸馏(DiD),一种无标签转换方法,仅通过匹配冻结的Softmax教师模型的检测器面向接口张量来训练线性注意力骨干。在DOTA-v1.5上,DiD显著优于现有基线,性能接近全监督训练的线性模型。4张GPU下约87分钟完成转换,线性化骨干使推理延迟降低约62%,峰值内存减少约49%。希望该方法为社区提供简单、无标签的高效模型复用路径,并推动未来架构转换中关注接口感知目标。

原文摘要 · Abstract (English)

While linear attention is a compelling mechanism for high-resolution object detection due to its reduced cost for global token mixing, converting the Softmax-attention ViT backbone of a trained detector into a linear-attention one is not a trivial drop-in replacement. Directly swapping the attention operator leads to severe performance degradation, and generic label-free distillation, though effective for classification, often fails on detection tasks. We argue that the central challenge is \textit{detector-interface preservation}: the converted backbone must reproduce the exact feature tensors expected by the fixed downstream detector, rather than merely imitating internal Softmax hidden states. To address this, we introduce Detector-Interface Distillation (DiD), a label-free conversion method that exclusively trains the linear-attention backbone by aligning detector-facing interface tensors with those of a frozen Softmax teacher. On DOTA-v1.5, DiD substantially outperforms established baselines and matches supervised, fully trained linear models. Adaptation completes in roughly 87 minutes on 4 GPUs, and the linearized backbone cuts inference latency by ~62% and peak memory by ~49%. We hope our findings offer the community a simple, label-free route to reusing trained Softmax detectors as efficient linear ones, and encourage interface-aware objectives in future architecture-conversion work.

目标检测线性注意力模型转换高效推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。