让大模型生成语义多样的回答，提升生成质量与多样性。
Semantic-guided Diverse Decoding for Large Language Model
- 在嵌入空间中通过方向引导和动态排斥实现语义级多样
- 在多个任务上提升Best-of-N覆盖率1.4%-5.2%，加速训练收敛15%
- 适合需要高质量多样化输出的场景，如对话系统与数据合成
大语言模型的多样化解码对需生成多个语义上不同响应的应用至关重要，但现有方法主要实现词汇层面而非语义层面的多样性。这严重限制了Best-of-N策略、基于群体的强化学习及数据合成的应用。尽管温度采样和多样束搜索通过调整词元分布或应用n-gram惩罚来改进，仍无法确保有意义的语义区分。本文提出语义引导多样化解码（SemDiD），直接在嵌入空间操作，通过三种互补机制平衡质量与多样性：正交方向引导、动态组间排斥及位置无关概率评估。利用自适应增益函数与约束优化，协调这些相互竞争的目标，确保满足质量阈值并实现最大语义差异。实验表明，SemDiD在多个任务中持续优于现有方法，使Best-of-N覆盖范围提升1.4%-5.2%，在强化学习人类反馈（RLHF）训练中加速收敛15%，准确率最高提升2.1%。
原文摘要 · Abstract (English)
Diverse decoding of large language models is crucial for applications requiring multiple semantically distinct responses, yet existing methods primarily achieve lexical rather than semantic diversity. This limitation significantly constrains Best-of-N strategies, group-based reinforcement learning, and data synthesis. While temperature sampling and diverse beam search modify token distributions or apply n-gram penalties, they fail to ensure meaningful semantic differentiation. We introduce Semantic-guided Diverse Decoding (SemDiD), operating directly in embedding space that balances quality with diversity through three complementary mechanisms: orthogonal directional guidance, dynamic inter-group repulsion, and position-debiased probability assessment. SemDiD harmonizes these competing objectives using adaptive gain functions and constraint optimization, ensuring both quality thresholds and maximal semantic differentiation. Experiments show SemDiD consistently outperforms existing methods, improving Best-of-N coverage by 1.4-5.2% across diverse tasks and accelerating RLHF training convergence by 15% while increasing accuracy by up to 2.1%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。