arXiv:2606.27984cs.LG2026-06被引 1

解决多模态数据缺失与时间错位问题,提升融合聚类精度。

Dual-Learning based Penalized Multi-Align Clustering for Multi-View Incomplete and Disorderly Data

论文配图:Dual-Learning based Penalized Multi-Align Clustering for Multi-View Incomplete and Disorderly Data
图 1 · 摘自论文原文
  • 双学习机制融合语义与结构信息,保持跨模态一致性。
  • 惩罚式多对多对齐,提升样本级配准准确率。
  • 适合处理传感器采样不均、数据错位的工业场景。

多模态特征融合可通过整合不同模态的互补信息,有效捕捉现实数据中的复杂模式。然而,在锅炉燃烧监控、设备故障诊断等应用中,传感器采样频率不一致、网络延迟等问题常导致模态缺失和时间异步,形成不完整且无序的多模态数据。现有方法虽在融合前对齐聚类中心,但仍存在两大缺陷:一是难以保证样本级的数据对齐精度;二是未解决不同类别间数据量显著差异的问题,影响后续融合效果。为此,本文提出一种基于双学习的惩罚式多对齐聚类模型(DLPMAC)。该模型通过双学习机制从各模态中学习先验知识,包括语义与结构信息,以在局部与全局层面保持跨模态的语义一致性和结构相似性。同时,惩罚式多对多对齐模块通过惩罚机制实现多对多数据对齐,允许一个样本与另一模态中多个样本形成数据对,从而提升数据对齐精度,并防止过度聚集现象。实验结果表明,DLPMAC在采样与聚类两个维度上均有效解决了数据对齐与融合挑战。

原文摘要 · Abstract (English)

Multimodal feature fusion can effectively capture complex patterns in real-world data by integrating complementary information from different modalities. However, in many applications, such as boiler combustion monitoring, equipment failure, inconsistent sensor sampling frequencies, and network delays often cause missing modalities and temporal asynchrony. These issues lead to incomplete and disorderly multimodal data. To address them, previous studies have proposed several data fusion methods that align cluster centers before fusion. However, these methods have two key limitations. First, they cannot guarantee accurate sample-level alignment of data pairs. Second, they do not address significant discrepancies in data sizes across different classes, which may affect subsequent fusion performance. To address these problems, we propose a dual-learning based penalized multi-align clustering model, named DLPMAC. The dual-learning mechanism enables the model to learn prior knowledge from each modality, including semantic and structural information. This helps preserve semantic consistency and structural similarity across modalities at both local and global levels. In addition, the penalized multi-align module performs multi-to-multi data alignment through a penalty mechanism. It allows one sample to form data pairs with different samples from other modalities, thereby improving data-pair alignment accuracy. The penalty mechanism also prevents data aggregation, avoiding the case where excessive samples are linked to a single sample. Experimental results demonstrate the effectiveness of DLPMAC in addressing data alignment and fusion challenges from both sampling and clustering perspectives.

多模态融合聚类数据对齐

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。