用先验注意力提升时间序列异常检测,尤其擅长发现时序与相位异常。
Pi-transformer: A prior-informed dual-attention model for multivariate time-series anomaly detection
- 设计双注意力机制:数据驱动与平滑先验注意力协同工作
- 在5个基准上达到顶尖或接近顶尖的F1分数,对时序/相位异常效果尤佳
- 可解释性强,适合工业系统中复杂多变量异常的实时监控
多变量时间序列中的异常往往源于时序上下文和通道间协同关系,而非孤立离群点。我们提出Pi-Transformer(先验感知变压器),一种具有双注意力路径的Transformer模型:数据驱动的序列注意力与平滑演化的先验注意力,后者编码尺度相关的自相似性与相位同步等时序不变特性。该先验提供幅值无关的时序参考,用于校准重建误差。训练时,采用重构目标与分歧项联合优化,促使两注意力一致但保持有意义区分,同时约束先验平滑演化,并轻量蒸馏至数据集级统计特征。推理阶段,模型融合对齐加权重构信号(能量)与不匹配信号,突出时间与相位扰动,生成单一判别得分。在SMD、MSL、SMAP、SWaT、PSM五个基准上,Pi-Transformer实现领先或具有竞争力的F1表现,尤其在时序与相位破坏类异常上优势显著。案例分析显示两路流互补,且在制度切换处具备可解释性检测能力。将先验注意力嵌入Transformer评分,为复杂多变量系统提供了校准且鲁棒的异常检测方法。
原文摘要 · Abstract (English)
Anomalies in multivariate time series often arise from temporal context and cross-channel coordination rather than isolated outliers. We present Pi-Transformer (Prior-Informed Transformer), a transformer with two attention pathways: data-driven series attention and a smoothly evolving prior attention that encodes temporal invariants such as scale-related self-similarity and phase synchrony. The prior provides an amplitude-insensitive temporal reference that calibrates reconstruction error. During training, we pair a reconstruction objective with a divergence term that encourages agreement between the two attentions while keeping them meaningfully distinct. The prior is regularised to evolve smoothly and is lightly distilled towards dataset-level statistics. At inference, the model combines an alignment-weighted reconstruction signal (Energy) with a mismatch signal that highlights timing and phase disruptions, and fuses them into a single score for detection. Across five benchmarks (SMD, MSL, SMAP, SWaT, and PSM), Pi-Transformer achieves state-of-the-art or highly competitive F1, with particular strength on timing and phase-breaking anomalies. Case analyses show complementary behaviour of the two streams and interpretable detections around regime changes. Embedding prior attention into transformer scoring yields a calibrated and robust approach to anomaly detection in complex multivariate systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。