探索大模型生成中不同解码方法的语义不确定性，发现结构化策略可兼顾多样与准确。
Semantic uncertainty in advanced decoding methods for LLM generation
- 对比多种解码方法，发现链式思维能提升输出多样性与置信度。
- 代码生成任务中，链式思维使Pass@2率提升48.8%，尽管与参考答案对齐度较低。
- 推测采样在摘要任务中表现优异，兼具高ROUGE分数与适中多样性，适合实用部署。
本研究探究大语言模型（LLM）在不同解码方法下输出的语义不确定性，重点关注推测采样和链式思维（CoT）等新兴技术。通过在问答、摘要生成和代码生成任务上的实验，分析不同解码策略对输出多样性和可靠性的影响。结果表明，虽然链式思维解码具有更高的语义多样性，但其预测熵更低，说明结构化探索可带来更自信且准确的输出。代码生成任务中，链式思维使Pass@2率提升48.8%，尽管与参考解对齐度较低；在摘要任务中，推测采样表现出色，获得更高ROUGE分数，同时保持中等语义多样性。研究挑战了多样性与准确性之间存在固有权衡的传统假设，证明合理设计的解码方法可在增强语义探索的同时维持或提升输出质量。该发现对需要兼顾可靠性和多样化解决方案的实际应用场景具有重要意义。
原文摘要 · Abstract (English)
This study investigates semantic uncertainty in large language model (LLM) outputs across different decoding methods, focusing on emerging techniques like speculative sampling and chain-of-thought (CoT) decoding. Through experiments on question answering, summarization, and code generation tasks, we analyze how different decoding strategies affect both the diversity and reliability of model outputs. Our findings reveal that while CoT decoding demonstrates higher semantic diversity, it maintains lower predictive entropy, suggesting that structured exploration can lead to more confident and accurate outputs. This is evidenced by a 48.8% improvement in code generation Pass@2 rates, despite lower alignment with reference solutions. For summarization tasks, speculative sampling proved particularly effective, achieving superior ROUGE scores while maintaining moderate semantic diversity. Our results challenge conventional assumptions about trade-offs between diversity and accuracy in language model outputs, demonstrating that properly structured decoding methods can increase semantic exploration while maintaining or improving output quality. These findings have significant implications for deploying language models in practical applications where both reliability and diverse solution generation are crucial.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。