提出新型注意力机制，精准建模时间序列的正负依赖关系。
Signed Dual Attention: Capturing Signed Dependencies in Time Series Forecasting
- 设计双消息传递结构，同时捕捉支持与对比信息。
- 无需额外参数，实现双头注意力的表达能力。
- 适合需建模符号依赖的时间序列任务，如金融、气象预测。
Transformer架构和注意力机制最初用于自然语言处理，现已成为广泛深度学习模型的核心，包括时间序列预测。然而，标准注意力机制隐含同质交互假设，难以建模具有正负依赖的数据（如时间序列）。本文提出签名双重注意力（Signed Dual Attention），一种无需额外参数的新注意力形式，能同时捕捉正负关系。该方法借鉴相关性结构，通过双消息传递机制，在单一共享模块内传播支持与对比信息，有效实现双头注意力的表达能力。该模块可无缝集成至现有架构，在需要符号关系建模时带来性能提升，为更高效且表达力更强的Transformer开辟新路径。
原文摘要 · Abstract (English)
Initially developed for natural language processing, Transformer architectures and attention mechanisms are now central to a wide range of deep learning models, including applications in time series forecasting. A standard attention mechanism, however, implicitly assumes homophilic interactions, limiting its ability to model data with positive and negative dependencies, such as time series. In this work, we introduce the Signed Dual Attention, a novel attention formulation that captures both positive and negative relational patterns without additional parameters. By leveraging a dual message-passing scheme inspired by correlation structures, Signed Dual Attention propagates both supportive and contrastive information within a single shared block, effectively achieving the expressiveness of two head attention without additional parameters. This module can be seamlessly integrated into existing architectures and can yield performance gains in certain situations, requiring signed relational modeling. This approach opens a pathway toward more expressive and parameter-efficient transformers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。