无需调参的文本生成采样方法，自动平衡熵与概率质量
Auxiliary-Hyperparameter-Free Sampling: Entropy Equilibrium for Text Generation
- 基于信息论设计，动态调节候选词集以平衡熵与概率分布
- 在多种温度设置下表现稳定，准确率与连贯性均达竞争水平
- 无需额外超参数，适合快速部署的生成任务
令牌采样策略显著影响大语言模型的文本生成质量。然而，现有方法引入额外超参数，需大量调优且增加部署复杂度。本文提出熵均衡采样（EES），一种受信息论启发的无辅助超参数方法，可动态调整候选集，平衡归一化熵与概率质量。我们在多种模型架构上对推理与生成任务进行了评估。结果表明，EES在不同温度设置下表现一致，兼具竞争力的准确率、连贯性与多样性。通过消除超参数调优需求，EES极大简化了部署流程并提升了性能。代码已公开于 https://github.com/shuanncai/EES。
原文摘要 · Abstract (English)
Token sampling strategies critically influence text generation quality in large language models (LLMs). However, existing methods introduce additional hyperparameters, requiring extensive tuning and complicating deployment. We present Entropy Equilibrium Sampling (EES), an auxiliary hyperparameter-free approach inspired by information theory that can dynamically adjust candidate sets by balancing normalized entropy with probability mass. We evaluate EES on both reasoning and generation tasks across a range of model architectures. Our results show that EES consistently performs well across temperature settings, delivering competitive accuracy and coherence while maintaining diversity. By eliminating the need for hyperparameter tuning, EES greatly simplifies deployment while improving performance. Code is available at https://github.com/shuanncai/EES
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。