解决大模型生成时反复重复的循环问题，让输出更自然多样。
LoopGuard: Breaking Self-Reinforcing Attention Loops via Dynamic KV Cache Intervention
- 通过动态修剪缓存中的重复内容，打破自我强化的注意力循环。
- 在基准测试中将重复发生率降低90个百分点以上，提升生成稳定性。
- 适用于长文本生成场景，特别适合追求高质量输出的研究者。
在长上下文生成任务中，我们发现解码过程会陷入持续重复的崩溃模式。这种退化由注意力模式坍缩引起：部分注意力头锁定历史记录的狭窄后缀，并因推理时的键值缓存重用而被强化。由于多数现有缓存策略依赖注意力重要性，这种坍缩会产生虚假的高分，导致缓存管理反而放大重复。为此，我们提出LoopBench，一个具备明确诱导条件和面向循环评估指标的基准，可量化重复严重性和生成不稳定性。基于此，我们设计了LoopGuard——一种轻量级、可插拔的缓存防护机制，能在线检测循环起点，并在固定缓存预算下剪枝重复尾部片段。实验表明，LoopGuard在LoopBench上使循环发生率下降超过90个百分点，同时恢复输出多样性并减少令牌浪费。
原文摘要 · Abstract (English)
Through systematic experiments on long-context generation, we observe a damaging failure mode in which decoding can collapse into persistent repetition loops. We find that this degeneration is driven by collapsed attention patterns, where a subset of heads locks onto a narrow suffix of the history, and is further stabilized by inference-time KV cache reuse. Crucially, since many existing KV cache policies rely on attention-based importance, this collapse can produce spuriously high scores for repetitive tokens, causing cache management to inadvertently amplify repetition. To study this phenomenon in a controlled and reproducible manner, we introduce LoopBench, a benchmark with explicit loop-inducing conditions and loop-oriented metrics that quantify repetition severity and generation instability beyond downstream task scores. Building on these insights, we propose LoopGuard, a lightweight, plug-in KV cache guard that detects loop onset online and disrupts the feedback cycle by pruning repetitive tail spans under a fixed cache budget. Experiments on LoopBench show that LoopGuard reduces loop incidence by over 90 percentage points, while restoring output diversity and reducing token waste.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。