提出轻量级实时生成式检测模型，支持直接生成类别名且速度超快。
RTGen: Real-Time Generative Detection Transformer
- 用统一框架联合解码视觉与文本特征，减少冗余结构。
- 在T4上达131.3帧/秒，比GenerateU快270倍以上。
- 无需CLIP或预训练语言模型，直接从检测标签学习命名。
尽管开放词汇目标检测器能泛化到未见类别，但推理时仍依赖预定义文本提示或分类头。近期生成式目标检测器通过结合自回归语言模型与检测主干，实现对每个检测对象的直接类别名生成。然而这种设计引入结构冗余并导致显著延迟。本文提出实时生成式检测变压器RTGen，采用简洁的编码器-解码器架构。特别地，引入新型区域-语言解码器（RL-Decoder），在统一框架内联合解码视觉与文本表示。文本侧采用有向无环图（DAG）组织，支持非自回归类别命名。得益于这些设计，RTGen-R34在T4 GPU上达到131.3 FPS，比GenerateU快超过270倍。此外，模型直接从检测标签学习生成类别名，无需依赖外部监督如CLIP或预训练语言模型，实现高效灵活的开放词汇检测。
原文摘要 · Abstract (English)
Although open-vocabulary object detectors can generalize to unseen categories, they still rely on predefined textual prompts or classifier heads during inference. Recent generative object detectors address this limitation by coupling an autoregressive language model with a detector backbone, enabling direct category name generation for each detected object. However, this straightforward design introduces structural redundancy and substantial latency. In this paper, we propose a Real-Time Generative Detection Transformer (RTGen), a real-time generative object detector with a succinct encoder-decoder architecture. Specifically, we introduce a novel Region-Language Decoder (RL-Decoder) that jointly decodes visual and textual representations within a unified framework. The textual side is organized as a Directed Acyclic Graph (DAG), enabling non-autoregressive category naming. Benefiting from these designs, RTGen-R34 achieves 131.3 FPS on T4 GPUs, over 270x faster than GenerateU. Moreover, our models learn to generate category names directly from detection labels, without relying on external supervision such as CLIP or pretrained language models, achieving efficient and flexible open-ended detection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。