用合成数据分组训练，让大模型生成更多样且高质量的回答。
Synthesize, Partition, then Adapt: Eliciting Diverse Samples from Foundation Models
- 通过数据归属分析将合成数据分组，针对性训练多个适配模型。
- 在HumanEval和MBPP任务中实现高多样性与高准确率的平衡。
- 适合需要多样化输出的代码生成与自然语言理解场景。
向用户呈现大模型生成的多样化响应对提升用户体验和满足不同偏好至关重要。然而，在使用贪婪采样时，如何在不牺牲准确性的情况下生成多个高质量且多样化的响应仍具挑战。本文提出一种新框架Synthesize-Partition-Adapt（SPA），利用众多领域中丰富的合成数据，通过影响函数等数据归属方法识别数据特征，将数据划分为多个子集，每个子集针对数据特定方面，训练多个优化于该子集的模型适配器。实验结果表明，该方法在代码生成领域的HumanEval和MBPP任务，以及多个自然语言理解任务中，有效提升了大模型响应的多样性并保持了高质性，展现出在各类应用中丰富用户交互体验的潜力。
原文摘要 · Abstract (English)
Presenting users with diverse responses from foundation models is crucial for enhancing user experience and accommodating varying preferences. However, generating multiple high-quality and diverse responses without sacrificing accuracy remains a challenge, especially when using greedy sampling. In this work, we propose a novel framework, Synthesize-Partition-Adapt (SPA), that leverages the abundant synthetic data available in many domains to elicit diverse responses from foundation models. By leveraging signal provided by data attribution methods such as influence functions, SPA partitions data into subsets, each targeting unique aspects of the data, and trains multiple model adaptations optimized for these subsets. Experimental results demonstrate the effectiveness of our approach in diversifying foundation model responses while maintaining high quality, showcased through the HumanEval and MBPP tasks in the code generation domain and several tasks in the natural language understanding domain, highlighting its potential to enrich user experience across various applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。