测试大模型在混杂对话中找回正确上下文的能力，提出新基准和记忆重建方法。
Reconstructing the Right Episode: Evaluating Interleaved Conversational Memory Beyond Long Context
- 设计新评测集SCALE-QA，检验模型在无分段对话中定位关键上下文的能力。
- 在3000个任务问答中，新方法比最强基线提升5.6至17.6个百分点准确率。
- 适合关注长对话记忆、上下文推理的模型开发者与研究者使用。
聊天助手的对话日益跨越多个话题且持续时间长，对记忆系统构成挑战。现有长上下文与记忆评测常暴露会话或主题边界，或询问直接个人记忆问题，未能体现更难的场景：在无分段的混杂对话中，系统需推断哪个早期对话片段使后续任务决策成立。本文提出SCALE-QA，一个面向无分割对话流的约束性任务问答基准，聚焦于对话片段完整性失效问题。数据集包含3000个经审核的问题，覆盖10个领域，采用确定性四选一评分机制，并配备确定性运行时构建器；实验使用全部3000个问题进行128k上下文测试，以及分层的400个诊断问题在1M上下文下测试。SCALE-QA问题为普通任务请求，其正确答案依赖于对话早期引入的因果相关证据。同时提出时间-语义交错记忆重建（TSIM）方法，通过层次化多视图记忆栈将对话流分割为连贯片段，并利用片段级摘要与聚类路由视图进行索引。实验表明，SCALE-QA挑战了强RAG基线与长上下文LLM；在三个开源与专有LLM后端中，TSIM在所有设置下均达到最高准确率，较最强基线提升5.6至17.6个百分点。
原文摘要 · Abstract (English)
Conversations with chat assistants increasingly span many topics in a single long-running thread, challenging memory systems. Existing long-context and memory benchmarks often expose session or topic boundaries, or probe direct personal-memory questions. These settings understate a harder assistant-memory regime: a flat mixed-topic thread where the system must infer which earlier episode makes a later task decision valid. We introduce SCALE-QA, a constraint-grounded task QA benchmark for flat unsegmented threads targeting episode integrity failure. The dataset contains 3,000 audited questions across 10 domains, uses deterministic four-way multiple-choice grading, and includes a deterministic runtime builder; experiments use all 3,000 questions through 128k and a stratified 400-question diagnostic at 1M. SCALE-QA questions are ordinary task-oriented requests whose correct answer depends on causally related evidence introduced earlier in the conversation. We also propose Temporal-Semantic Interleaved Memory Reconstruction (TSIM), which segments the turn stream into coherent episodes and indexes them through a hierarchical multi-view memory stack with deterministic episode-level summary and cluster-routing views. Experiments show that SCALE-QA challenges strong RAG baselines and long-context LLMs alike; across three open-source and proprietary LLM backends, TSIM achieves the highest accuracy in every backend setting, gaining 5.6-17.6 accuracy points over the strongest corresponding baseline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。