arXiv:2509.23316cs.CV2025-09被引 6

提出C3-OWD框架,统一提升目标检测在复杂环境下的鲁棒性与未知类别泛化能力。

C3-OWD: A Curriculum Cross-modal Contrastive Learning Framework for Open-World Detection

  • 分阶段训练:先用RGBT数据增强鲁棒性,再通过视觉语言对齐提升泛化能力
  • 在FLIR、OV-COCO、OV-LVIS上分别取得80.1 AP⁵⁰、48.6 AP⁵⁰ₙₒᵥₑₗ、35.7 mAPᵣ
  • 采用EMA机制防止遗忘,适合需要跨模态与开放世界检测的应用

目标检测在封闭集设置下已取得显著进展,但实际部署仍受限于两个挑战:对未见类别的泛化能力差,以及在恶劣条件下的鲁棒性不足。以往研究分别探索了这些问题:可见光-红外检测提升了鲁棒性但缺乏泛化能力,而开放世界检测利用视觉-语言对齐策略增强了类别多样性,却在极端环境下表现不佳。这种权衡导致鲁棒性与多样性难以兼得。为此,我们提出C3-OWD,一种课程式跨模态对比学习框架,融合二者优势。第一阶段通过RGBT数据预训练提升鲁棒性,第二阶段通过视觉-语言对齐增强泛化能力。为防止两阶段间灾难性遗忘,引入指数移动平均(EMA)机制,理论上保证前阶段性能不下降,参数偏移有界且函数一致。在FLIR、OV-COCO和OV-LVIS数据集上的实验表明,该方法在鲁棒性与多样性评估中均表现优异:在FLIR上达到80.1 AP⁵⁰,OV-COCO上新类别达48.6 AP⁵⁰ₙₒᵥₑₗ,OV-LVIS上为35.7 mAPᵣ。代码开源:https://github.com/justin-herry/C3-OWD.git。

原文摘要 · Abstract (English)

Object detection has advanced significantly in the closed-set setting, but real-world deployment remains limited by two challenges: poor generalization to unseen categories and insufficient robustness under adverse conditions. Prior research has explored these issues separately: visible-infrared detection improves robustness but lacks generalization, while open-world detection leverages vision-language alignment strategy for category diversity but struggles under extreme environments. This trade-off leaves robustness and diversity difficult to achieve simultaneously. To mitigate these issues, we propose \textbf{C3-OWD}, a curriculum cross-modal contrastive learning framework that unifies both strengths. Stage~1 enhances robustness by pretraining with RGBT data, while Stage~2 improves generalization via vision-language alignment. To prevent catastrophic forgetting between two stages, we introduce an Exponential Moving Average (EMA) mechanism that theoretically guarantees preservation of pre-stage performance with bounded parameter lag and function consistency. Experiments on FLIR, OV-COCO, and OV-LVIS demonstrate the effectiveness of our approach: C3-OWD achieves $80.1$ AP$^{50}$ on FLIR, $48.6$ AP$^{50}_{\text{Novel}}$ on OV-COCO, and $35.7$ mAP$_r$ on OV-LVIS, establishing competitive performance across both robustness and diversity evaluations. Code available at: https://github.com/justin-herry/C3-OWD.git.

目标检测跨模态开放世界鲁棒性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。