arXiv:2606.12411cs.CLcs.LG2026-06中稿 · ICML

提出增量式对话压缩方法,让长对话更高效稳定。

Context-Driven Incremental Compression for Multi-Turn Dialogue Generation

论文配图:Context-Driven Incremental Compression for Multi-Turn Dialogue Generation
图 1 · 摘自论文原文
  • 将对话视为可更新的上下文线程,共享跨轮记忆
  • 支持数百轮对话下稳定延迟与困惑度
  • 适合需要长程对话建模的智能客服场景

现代对话系统在每轮对话中需处理不断增长的历史信息,导致注意力和编码开销随对话长度增加。简单截断或摘要会降低语义保真度,现有压缩方法缺乏跨轮记忆共享与修正机制,导致长对话中信息丢失与误差累积。本文重新审视对话压缩在动态对话中的脆弱性,提出上下文驱动的增量压缩(C-DIC):将对话视为交错的上下文线程,将可修订的每线程压缩状态存于单一紧凑对话记忆中。每轮通过轻量级检索、修正与写回循环,实现跨轮信息共享并更新过时记忆,稳定长程行为。此外,将截断反向传播时间序列(TBPTT)适配至多轮场景,学习跨轮依赖而无需全历史反向传播。在长对话基准上的大量实验表明,C-DIC在性能与效率上均优于基线;尤其在数百轮对话中保持稳定的推理延迟与困惑度,为高质量对话建模提供可扩展路径。

原文摘要 · Abstract (English)

Modern conversational agents condition on an ever-growing dialogue history at each turn, incurring redundant attention and encoding costs that grow with conversation length. Naive truncation or summarization degrades fidelity, while existing context compressors lack cross-turn memory sharing or revision, causing information loss and compounding errors in long dialogues. We revisit the context compression under conversational dynamics and empirically present its fragility. To improve both efficiency and robustness, we introduce Context-Driven Incremental Compression (C-DIC), which treats a conversation as interleaved contextual threads and stores revisable per-thread compression states in a single, compact dialogue memory. At each turn, a lightweight retrieve, revise, and write-back loop shares information across turns and updates stale memories, stabilizing long-horizon behavior. In addition, we adapt truncated backpropagation-through-time (TBPTT) to our multi-turn setting, learning cross-turn dependencies without full-history backpropagation. Extensive experiments on long-form dialogue benchmarks demonstrate superior performance and efficiency of C-DIC; notably, C-DIC shows stable inference latency and perplexity over hundreds of dialogue turns, supporting a scalable path to high-quality dialogue modeling.

对话压缩长对话增量学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。