ASTS改进典型性采样，让大模型生成更流畅多样且不重复。
Advancing Decoding Strategies: Enhancements in Locally Typical Sampling for LLMs
- 动态熵阈值+多目标评分，智能调节生成策略
- 在故事生成和摘要任务中降低重复率，提升语义对齐
- 适合追求高质量生成效果的开发者与研究者
本章探讨大语言模型解码策略的进展，聚焦于增强局部典型性采样（LTS）算法。传统方法如top-k和核采样难以平衡文本生成的流畅性、多样性和连贯性。为此，提出自适应语义感知典型性采样（ASTS），引入动态熵阈值、多目标评分及奖励-惩罚调整机制。ASTS在保持计算效率的同时，确保生成内容上下文连贯且多样化。在多个基准测试中评估，包括故事生成和抽象摘要任务，使用困惑度、MAUVE和多样性得分等指标。实验结果表明，ASTS相比现有采样技术显著减少重复，增强语义一致性并提升流畅性。
原文摘要 · Abstract (English)
This chapter explores advancements in decoding strategies for large language models (LLMs), focusing on enhancing the Locally Typical Sampling (LTS) algorithm. Traditional decoding methods, such as top-k and nucleus sampling, often struggle to balance fluency, diversity, and coherence in text generation. To address these challenges, Adaptive Semantic-Aware Typicality Sampling (ASTS) is proposed as an improved version of LTS, incorporating dynamic entropy thresholding, multi-objective scoring, and reward-penalty adjustments. ASTS ensures contextually coherent and diverse text generation while maintaining computational efficiency. Its performance is evaluated across multiple benchmarks, including story generation and abstractive summarization, using metrics such as perplexity, MAUVE, and diversity scores. Experimental results demonstrate that ASTS outperforms existing sampling techniques by reducing repetition, enhancing semantic alignment, and improving fluency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。