arXiv:2512.20892cs.CV2025-12被引 1

用特征空间注入提升跨模态船舶识别,无需大量配对数据

Beyond Weight Adaptation: Feature-Space Domain Injection for Cross-Modal Ship Re-Identification

  • 在特征空间而非权重空间优化,保持模型不变
  • 仅用154万参数达57.9% mAP,性能超现有方法
  • 适合资源有限但需高精度跨模态识别的场景

跨模态船舶重识别(CMS Re-ID)对全天候海上目标追踪至关重要,但受制于显著的模态差异。主流方法依赖显式模态对齐,严重依赖大规模配对数据预训练。基于柏拉图表征假说,本文探索视觉基础模型(VFMs)在弥合模态差距中的潜力。针对现有通用参数高效微调(PEFT)方法在低容量模型上表现不佳的问题,提出一种新策略——领域表征注入(DRI)。该方法冻结完整VFM以保留通用知识,设计轻量可学习的偏移编码器,从原始输入中提取富含模态与身份属性的领域表征。通过调制器根据中间层上下文信息动态转换这些表征,并以加性融合方式注入中间层,从而在不修改预训练权重的前提下,动态重塑特征分布以适应下游任务。实验表明,该方法在保持极小可训练参数量的同时达到最优性能:在HOSS-ReID数据集上,分别使用154万和705万参数时,mAP达到57.9%和60.5%。代码已开源。

原文摘要 · Abstract (English)

Cross-Modality Ship Re-Identification (CMS Re-ID) is critical for achieving all-day and all-weather maritime target tracking, yet it is fundamentally challenged by significant modality discrepancies. Mainstream solutions typically rely on explicit modality alignment strategies; however, this paradigm heavily depends on constructing large-scale paired datasets for pre-training. To address this, grounded in the Platonic Representation Hypothesis, we explore the potential of Vision Foundation Models (VFMs) in bridging modality gaps. Recognizing the suboptimal performance of existing generic Parameter-Efficient Fine-Tuning (PEFT) methods that operate within the weight space, particularly on limited-capacity models, we shift the optimization perspective to the feature space and propose a novel PEFT strategy termed Domain Representation Injection (DRI). Specifically, while keeping the VFM fully frozen to maximize the preservation of general knowledge, we design a lightweight, learnable Offset Encoder to extract domain-specific representations rich in modality and identity attributes from raw inputs. Guided by the contextual information of intermediate features at different layers, a Modulator adaptively transforms these representations. Subsequently, they are injected into the intermediate layers via additive fusion, dynamically reshaping the feature distribution to adapt to the downstream task without altering the VFM's pre-trained weights. Extensive experimental results demonstrate the superiority of our method, achieving State-of-the-Art (SOTA) performance with minimal trainable parameters. For instance, on the HOSS-ReID dataset, we attain 57.9\% and 60.5\% mAP using only 1.54M and 7.05M parameters, respectively. The code is available at https://github.com/TingfengXian/DRI.

跨模态识别特征空间参数高效

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。