用动态掩码和半监督学习提升不规则序列数据的意图预测效果
STaRFormer: Semi-Supervised Task-Informed Representation Learning via Dynamic Attention-Based Regional Masking for Sequential Data
- 基于动态注意力区域掩码，自适应处理不规则采样序列
- 在56个不同数据集上超越现有方法，显著提升意图识别准确率
- 适合处理真实场景中非平稳、传感器受限的时序数据
理解用户意图对情境感知与上下文决策至关重要。针对车辆周边智能设备用户的意图预测问题，本文建模序列时空数据。然而，现实场景中环境因素与传感器限制导致数据非平稳且采样不规则，带来巨大挑战。为此，我们提出STaRFormer——一种基于Transformer的通用序列建模框架。该方法结合新型动态注意力区域掩码机制与半监督对比学习范式，增强任务相关的潜在表征。在涵盖非平稳与不规则采样数据的56个不同数据集上进行的全面实验表明，STaRFormer在多种类型、任务、领域、序列长度、训练样本量和应用场景下均有效，显著优于当前最先进方法。
原文摘要 · Abstract (English)
Understanding user intent is essential for situational and context-aware decision-making. Motivated by a real-world scenario, this work addresses intent predictions of smart device users in the vicinity of vehicles by modeling sequential spatiotemporal data. However, in real-world scenarios, environmental factors and sensor limitations can result in non-stationary and irregularly sampled data, posing significant challenges. To address these issues, we propose STaRFormer, a Transformer-based approach that can serve as a universal framework for sequential modeling. STaRFormer utilizes a new dynamic attention-based regional masking scheme combined with a novel semi-supervised contrastive learning paradigm to enhance task-specific latent representations. Comprehensive experiments on 56 datasets varying in types (including non-stationary and irregularly sampled), tasks, domains, sequence lengths, training samples, and applications demonstrate the efficacy of STaRFormer, achieving notable improvements over state-of-the-art approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。