通过设计中间生成规范提升大模型输出多样性。
Where You Inject Diversity Matters: A Unified Framework for Diverse Generation
- 在生成过程的中间阶段注入多样化规范，再据此生成最终结果。
- 跨五项任务、四类模型，多样性显著优于现有方法。
- 适合需要丰富输出的对话、写作等开放生成场景。
开放生成任务常需一组语义上不同的输出，但大语言模型易产生相似内容。现有测试时多样性方法在生成不同阶段起效不一，但其设计选择对输出多样性的影响尚不明确。本文提出一个统一框架，按生成过程中引入多样性的来源进行分类，并引入传输分数衡量源变化传递到最终输出的有效性。基于该框架，我们提出全自动的规格级生成方法：先生成多样化的中间规格，再以此为条件生成最终响应。在五个开放生成任务和四种主干模型上，规格级注入在保持相近质量的前提下，显著提升了输出多样性。分析表明，成功的多样性注入依赖于源的多样性及其向输出的有效传递，强调了源设计与源到输出实现是构建更多样化生成系统的关键杠杆。
原文摘要 · Abstract (English)
Open-ended generation tasks often require a set of meaningfully different outputs, yet large language models often produce similar generations. Existing test-time diversity methods operate at different stages of generation with varying effectiveness, but it remains unclear what design choices lead to meaningful diversity in the output. We introduce a framework that characterizes test-time diverse generation methods by the diversity source introduced during generation and provide a transmission score for measuring how effectively variation in the source reaches the final output. Guided by this framework, we propose fully automated specification-level generation methods that first generate diverse intermediate specifications and then condition on them to produce final responses. Across five open-ended tasks and four backbone models, specification-level injection improves output diversity over test-time baselines while maintaining comparable quality. Our analysis shows that successful diversity injection depends on both the diversity of the sources and their transmission to the output, highlighting source design and source-to-output realization as two key levers for building more diverse generation systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。