针对开放词汇目标检测的低比特量化难题，提出多模态对齐训练方法。
QATMA: Quantization-Aware Training with Multimodal Alignment for Open-Vocabulary Object Detection
- 分阶段渐进量化+文本锚定相似性蒸馏，保持跨模态与模内对齐。
- 在LVIS和COCO零样本测试中，低比特下提升4.3~7.6点AP。
- 适用于低资源部署的开放词汇检测，尤其适合极端量化场景。
量化开放词汇目标检测（OVOD）模型可降低内存与计算开销，但极低比特量化会严重破坏跨模态（区域-文本）与模内（区域-区域）对齐关系。这一多模态退化是现有封闭词汇检测量化方法无法解决的独特挑战。为此，我们提出首个面向OVOD的多模态感知、架构无关的量化感知训练框架QATMA。其包含两个核心组件：(i) 课程式量化训练（Curriculum QAT），按功能模块划分检测器，逐步扩展量化范围以抑制误差累积并保障稳定优化；(ii) 文本锚定成对相似性蒸馏（Text-anchored Pairwise Similarity Distillation），通过联合嵌入空间中的成对余弦相似度，将全精度教师模型的区域-文本与区域-区域对齐关系迁移至学生模型。在LVIS与COCO零样本基准上的实验表明，QATMA在极低比特设置下显著优于现有量化基线，在两种数据集上分别实现最高4.3和7.6点的平均精度（AP）提升。
原文摘要 · Abstract (English)
Quantizing open-vocabulary object detection (OVOD) models reduces their memory and computational costs, but extremely low-bit quantization severely degrades both cross-modal (region-text) and intra-modal (region-region) alignments. This multimodal degradation is a unique challenge that prior quantization methods for closed-vocabulary detectors fail to resolve. To overcome this, we propose Quantization-Aware Training with Multimodal Alignment (QATMA), the first multimodal-aware and architecture-agnostic QAT framework tailored for OVOD. QATMA integrates two key components: (i) Curriculum QAT, which partitions the detector by functional roles and progressively expands the quantization scope to suppress error accumulation and ensure stable optimization; and (ii) Text-anchored Pairwise Similarity Distillation, which transfers both region-text and region-region alignments from a full-precision teacher model via pairwise cosine similarities in the joint embedding space. Experimental results on LVIS and COCO zero-shot benchmarks demonstrate that QATMA significantly outperforms existing QAT baselines under extremely low-bit settings, achieving gains of up to 4.3 and 7.6 AP, respectively.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。