arXiv:2512.20092cs.CL2025-12被引 8

用强化学习筛选对话记忆,让聊天机器人更准地处理长时间对话中的时间逻辑。

Memory-T1: Reinforcement Learning for Temporal Reasoning in Multi-session Agents

  • 通过粗筛+强化学习精选,动态选择与时间相关的对话片段。
  • 在128k token长对话下仍稳定表现,比14B模型高10.2分。
  • 适合需要处理长期对话的智能助手、客服系统等场景。

长时多轮对话中的时间推理是对话智能体的关键能力。然而,现有方法在对话历史变长且含噪声时,难以准确识别时间相关信息,导致推理性能下降。为此,我们提出Memory-T1框架,利用强化学习(RL)训练时间感知的记忆选择策略。该框架采用由粗到精的策略:先通过时间和相关性过滤器将对话历史缩减为候选集,再由RL代理精准选出证据会话。RL训练采用多层级奖励函数,优化答案准确性、证据可追溯性及时间一致性。其中,时间一致性奖励通过会话级(时间顺序接近度)和话语级(时间忠实度)双重评估,提供密集信号以解决细微的时间歧义。在Time-Dialog基准上,Memory-T1使7B模型达到67.0%的整体得分,刷新开源模型新纪录,并超越14B基线10.2个百分点。消融实验表明,时间一致性和证据可追溯性奖励共同带来15.0%的性能提升。此外,该框架在长达128k tokens的对话中仍保持鲁棒性,而基线模型已崩溃。代码与数据集已公开于https://github.com/Elvin-Yiming-Du/Memory-T1/

原文摘要 · Abstract (English)

Temporal reasoning over long, multi-session dialogues is a critical capability for conversational agents. However, existing works and our pilot study have shown that as dialogue histories grow in length and accumulate noise, current long-context models struggle to accurately identify temporally pertinent information, significantly impairing reasoning performance. To address this, we introduce Memory-T1, a framework that learns a time-aware memory selection policy using reinforcement learning (RL). It employs a coarse-to-fine strategy, first pruning the dialogue history into a candidate set using temporal and relevance filters, followed by an RL agent that selects the precise evidence sessions. The RL training is guided by a multi-level reward function optimizing (i) answer accuracy, (ii) evidence grounding, and (iii) temporal consistency. In particular, the temporal consistency reward provides a dense signal by evaluating alignment with the query time scope at both the session-level (chronological proximity) and the utterance-level (chronological fidelity), enabling the agent to resolve subtle chronological ambiguities. On the Time-Dialog benchmark, Memory-T1 boosts a 7B model to an overall score of 67.0\%, establishing a new state-of-the-art performance for open-source models and outperforming a 14B baseline by 10.2\%. Ablation studies show temporal consistency and evidence grounding rewards jointly contribute to a 15.0\% performance gain. Moreover, Memory-T1 maintains robustness up to 128k tokens, where baseline models collapse, proving effectiveness against noise in extensive dialogue histories. The code and datasets are publicly available at https://github.com/Elvin-Yiming-Du/Memory-T1/

时间推理强化学习长对话记忆机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。