arXiv:2603.21724cs.LGcs.AI2026-03被引 1

用模糊推理替代注意力机制,提升时序预测的准确性与可解释性

FISformer: Replacing Self-Attention with a Fuzzy Inference System in Transformer Models for Time Series Forecasting

  • 以模糊推理系统取代传统注意力,按特征维度计算关系强度
  • 在多个基准数据集上超越现有Transformer模型,噪声鲁棒性强
  • 适合需要可解释性与时序不确定性建模的应用场景

Transformers在时序预测中取得显著进展,但其依赖确定性的点积注意力,难以建模不确定性及多变量时间维度间的非线性依赖。为此,我们提出FISFormer,一种基于模糊推理系统的Transformer,用FIS交互机制替代传统注意力。每个查询-键对在各特征维度上进行模糊推理,通过可学习的隶属函数与规则推理估算令牌间关系强度。这些由模糊推理生成的交互权重捕捉不确定性,并提供可解释的连续映射。沿令牌轴应用softmax归一化后,与对应值特征逐元素相乘,得到最终上下文增强的令牌表示。该设计融合了模糊逻辑的可解释性与不确定性建模能力,以及Transformer的表征能力。在多个基准数据集上的大量实验表明,FISFormer在预测精度、噪声鲁棒性和可解释性方面均优于现有先进Transformer变体,确立模糊推理作为传统注意力的有效替代方案。

原文摘要 · Abstract (English)

Transformers have achieved remarkable progress in time series forecasting, yet their reliance on deterministic dot-product attention limits their capacity to model uncertainty and nonlinear dependencies across multivariate temporal dimensions. To address this limitation, we propose FISFormer, a Fuzzy Inference System-driven Transformer that replaces conventional attention with a FIS Interaction mechanism. In this framework, each query-key pair undergoes a fuzzy inference process for every feature dimension, where learnable membership functions and rule-based reasoning estimate token-wise relational strengths. These FIS-derived interaction weights capture uncertainty and provide interpretable, continuous mappings between tokens. A softmax operation is applied along the token axis to normalize these weights, which are then combined with the corresponding value features through element-wise multiplication to yield the final context-enhanced token representations. This design fuses the interpretability and uncertainty modeling of fuzzy logic with the representational power of Transformers. Extensive experiments on multiple benchmark datasets demonstrate that FISFormer achieves superior forecasting accuracy, noise robustness, and interpretability compared to state-of-the-art Transformer variants, establishing fuzzy inference as an effective alternative to conventional attention mechanisms.

时序预测模糊推理Transformer可解释性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。