arXiv:2605.10130cs.CV2026-05中稿 · CVPR

首个面向热成像的开源词汇目标检测模型,用语言引导跨模态蒸馏。

Thermal-Det: Language-Guided Cross-Modal Distillation for Open-Vocabulary Thermal Object Detection

论文配图:Thermal-Det: Language-Guided Cross-Modal Distillation for Open-Vocabulary Thermal Object Detection
图 1 · 摘自论文原文
  • 用合成数据+去噪文本构建百万级热图-文本对训练集
  • 通过冻结的RGB教师模型提供伪监督,提升热图像检测性能2-4%
  • 适合需要语言驱动热成像感知的智能监控与自动驾驶场景

现有开放词汇检测器主要针对RGB图像,难以泛化到热成像领域,因低纹理和发射率变化挑战了基于RGB的语义理解。本文提出Thermal-Det,首个由大语言模型(LLM)监督的、专为热成像设计的开放词汇检测器。为支持大规模训练,我们通过将GroundingCap-1M转换至热域并过滤掉RGB特有词汇,构建了一个包含超百万张带边界框、定位文本和详细描述的热成像数据集。Thermal-Det联合优化检测、文本生成与跨模态蒸馏任务。一个冻结的RGB教师模型为成对但未标注的RGB-热数据提供几何与语义伪监督,实现无需人工标注的开放词汇知识迁移。模型还引入热-文对齐头进行文本校准,以及模态融合交叉注意力模块实现双模态推理。不同于传统领域自适应方法,该检测器全量微调以内化热成像对比模式,同时保持语言对齐。在公开基准上的实验显示,其相比现有开放词汇检测器稳定提升2-4%的平均精度(AP),为可扩展的语言驱动热成像感知奠定了坚实基础。

原文摘要 · Abstract (English)

Existing open-vocabulary detectors focus on RGB images and fail to generalize to thermal imagery, where low texture and emissivity variations challenge RGB-based semantics. We present Thermal-Det, the first large language model (LLM) supervised open-vocabulary detector tailored for thermal images. To enable large-scale training, we develop a synthetic dataset by converting GroundingCap-1M into the thermal domain and filtering captions to remove RGB-specific terms, yielding over one million thermally aligned samples with bounding boxes, grounding texts, and detailed captions. Thermal-Det jointly optimizes detection, captioning, and cross-modal distillation objectives. A frozen RGB teacher provides geometric and semantic pseudo-supervision for paired but unlabeled RGB-thermal data, transferring open-vocabulary knowledge without manual annotation. The model further employs a Thermal-Text Alignment Head for text calibration and a Modality-Fused Cross-Attention module for dual-modality reasoning. Unlike prior domain-adaptation methods, the detector is fully fine-tuned to internalize thermal contrast patterns while preserving language alignment. Experiments on public benchmarks show consistent 2-4% AP gains over existing open-vocabulary detectors, establishing a strong foundation for scalable, language-driven thermal perception.

热成像开放词汇跨模态语言引导

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。