让检索结果更精准：通过交叉注意力筛选相关历史数据提升时序预测
Not All Retrievals are Useful: Cross-Attention for Input-Aware RAG in Time Series Forecasting
- 用查询-检索交叉注意力机制判断哪些历史样本相关
- 在多种模型上均提升零样本预测性能，且不依赖手动调参k值
- 适合需要可靠外源知识的时序预测场景
检索增强生成（RAG）通过利用外部知识库提升零样本时序预测能力，但现有方法在融合检索样本时忽略了查询层面的相关性。我们指出，并非所有检索结果都有效，无关样本会降低性能。为此，提出Cross-RAG框架，通过查询-检索交叉注意力机制，有选择地关注与查询相关的检索样本。该设计联合建模三类信息：查询本身、检索样本及其相互关系。特别地，输入感知结构使Cross-RAG在检索样本数量k增加时仍保持稳定，而此前无交叉注意力的方法需精细调节k以避免无关样本带来的性能下降。大量实验表明，Cross-RAG在多个TSFM骨干模型和不同RAG方法中均持续提升零样本预测性能，额外分析也验证了其在各类检索场景下的有效性。代码已开源。
原文摘要 · Abstract (English)
Retrieval-augmented generation (RAG) enhances zero-shot time series (TS) forecasting by leveraging external knowledge bases, yet existing approaches overlook input-level relevance when fusing retrieved samples with the query. We argue that not all retrievals are equally useful, and irrelevant ones can degrade performance. To this end, we propose Cross-RAG, a zero-shot RAG-based forecasting framework that selectively attends to query-relevant retrieved samples via query--retrieval cross-attention. By modeling input-level relevance between the query and retrieved samples, Cross-RAG jointly incorporates three sources of information: 1) the query itself, 2) the retrieved samples, and 3) their relational interactions. In particular, this input-aware design enables Cross-RAG to remain stable as the number of retrieved samples $k$ grows, whereas prior methods without cross-attention require careful $k$ tuning to avoid degradation from irrelevant retrievals. Extensive experiments demonstrate that Cross-RAG consistently improves zero-shot forecasting performance across multiple TSFM backbones and various RAG methods, with additional analyses confirming its effectiveness across various retrieval scenarios. Code is available at https://github.com/seunghan96/cross-rag/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。