用熵值动态调整对话上下文，提升大模型多轮对话表现
ERGO: Entropy-guided Resetting for Generation Optimization in Multi-turn Language Models
- 通过香农熵监测生成不确定性，触发自适应上下文重置
- 多轮任务中性能平均提升56.6%，可靠性下降35.3%
- 适合需要稳定对话能力的智能助手、客服系统
大型语言模型在多轮对话中因信息逐步呈现导致性能显著下降，严重影响实际应用。我们提出，模型不确定性骤增是对话错位的信号，并据此设计了ERGO（熵引导的生成优化重置机制）。该方法持续通过香农熵量化下一词分布的内部不确定性，当熵值突增时触发自适应提示整合。将不确定性视为可利用信号而非干扰，使模型能感知并响应语言与建模中的自然波动。在逐步揭示指令的多轮任务中，ERGO相比标准基线平均性能提升56.6%，峰值能力提高24.7%，不可靠性降低35.3%，证明了感知不确定性的干预能同时提升对话AI的准确性和稳定性。
原文摘要 · Abstract (English)
Large Language Models (LLMs) suffer significant performance degradation in multi-turn conversations when information is presented incrementally. Given that multi-turn conversations characterize everyday interactions with LLMs, this degradation poses a severe challenge to real world usability. We hypothesize that abrupt increases in model uncertainty signal misalignment in multi-turn LLM interactions, and we exploit this insight to dynamically realign conversational context. We introduce ERGO (Entropy-guided Resetting for Generation Optimization), which continuously quantifies internal uncertainty via Shannon entropy over next token distributions and triggers adaptive prompt consolidation when a sharp spike in entropy is detected. By treating uncertainty as a first class signal rather than a nuisance to eliminate, ERGO embraces variability in language and modeling, representing and responding to uncertainty. In multi-turn tasks with incrementally revealed instructions, ERGO yields a 56.6% average performance gain over standard baselines, increases aptitude (peak performance capability) by 24.7%, and decreases unreliability (variability in performance) by 35.3%, demonstrating that uncertainty aware interventions can improve both accuracy and reliability in conversational AI.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。