arXiv:2503.09873cs.CV2025-03中稿 · the IEEE Transacti…被引 2

多传感器目标分类中,通过频域感知分解与跨模态对齐提升识别准确率。

FDCT: Frequency-Aware Decomposition and Cross-Modal Token-Alignment for Multi-Sensor Target Classification

  • 分域提取特征,构建统一离散令牌空间缩小模态差异。
  • 引入稀疏约束对齐模块,在四组数据集上性能超越主流融合方法。
  • 适合解决传感器异构、错位问题的多模态目标识别场景。

在自动目标识别系统中,由于环境条件、CMOS芯片噪声、遮挡、视差和传感器错位等因素,单个传感器难以捕捉细粒度判别特征。因此,多传感器图像融合成为有效解决方案。然而,多模态图像传感器存在异构性,且在领域和粒度上存在差距,同时复杂背景、光照变化和未控传感器设置会导致图像错位。为此,本文提出对多源图像数据进行分解、对齐与融合以实现目标分类。从各传感器数据中提取领域特异性与领域不变性特征,并构建跨传感器共享的统一离散令牌(UDT)空间,以缓解领域与粒度差异。进一步设计对齐模块,克服多传感器间错位问题,强化UDT空间的判别表示。该模块引入稀疏性约束,增强跨模态表征能力并提升对不同传感器设置的鲁棒性。在四个多传感器ATR数据集上,本方法在分类性能上优于单模态分类器及多个前沿多模态融合算法。

原文摘要 · Abstract (English)

In automatic target recognition (ATR) systems, sensors may fail to capture discriminative, fine-grained detail features due to environmental conditions, noise created by CMOS chips, occlusion, parallaxes, and sensor misalignment. Therefore, multi-sensor image fusion is an effective choice to overcome these constraints. However, multi-modal image sensors are heterogeneous and have domain and granularity gaps. In addition, the multi-sensor images can be misaligned due to intricate background clutters, fluctuating illumination conditions, and uncontrolled sensor settings. In this paper, to overcome these issues, we decompose, align, and fuse multiple image sensor data for target classification. We extract the domain-specific and domain-invariant features from each sensor data. We propose to develop a shared unified discrete token (UDT) space between sensors to reduce the domain and granularity gaps. Additionally, we develop an alignment module to overcome the misalignment between multi-sensors and emphasize the discriminative representation of the UDT space. In the alignment module, we introduce sparsity constraints to provide a better cross-modal representation of the UDT space and robustness against various sensor settings. We achieve superior classification performance compared to single-modality classifiers and several state-of-the-art multi-modal fusion algorithms on four multi-sensor ATR datasets.

多传感器融合目标识别跨模态对齐特征分解

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。