通过可控探索提升大模型输出多样性，不增加延迟。
Improving Linguistic Diversity of Large Language Models with Possibility Exploration Fine-Tuning
- 提出无任务依赖的多样性增强框架，无需额外计算开销。
- 相同提示下可生成多个语义多样响应，相似度显著降低。
- 有效缓解对话系统中的群体偏见，适合低延迟场景。
尽管大语言模型在模拟人类能力方面取得显著进展，但其输出语言多样性下降的问题日益突出，导致观点同质化及特定群体代表性不足。现有微调与提示方法多针对特定任务，或带来显著计算成本与延迟，难以应用于对延迟敏感的场景（如聊天机器人、虚拟助手）。本文提出可能性探索微调（PEFT），一种无需增加延迟或计算成本的任务无关框架，使模型在相同提示下可同时生成多个可控可能性的多样化响应。对话与故事生成任务实验表明，PEFT 显著提升输出多样性，候选响应间相似度降低。由于侧重语义多样性而非词汇多样性，该方法还能有效减少对话系统中的群体偏见。代码与数据集已开源：https://github.com/mailong25/peft_diversity。
原文摘要 · Abstract (English)
While Large Language Models (LLMs) have made significant strides in replicating human-like abilities, there are concerns about a reduction in the linguistic diversity of their outputs. This results in the homogenization of viewpoints and perspectives, as well as the underrepresentation of specific demographic groups. Although several fine-tuning and prompting techniques have been suggested to tackle the issue, they are often tailored to specific tasks or come with a substantial increase in computational cost and latency. This makes them challenging to apply to applications that demand very low latency, such as chatbots and virtual assistants. We propose Possibility Exploration Fine-Tuning (PEFT), a task-agnostic framework that enhances the text diversity of LLMs without increasing latency or computational cost. Given the same prompt, models fine-tuned with PEFT can simultaneously generate multiple diverse responses, each corresponding with a controllable possibility number. Experiments on dialogue and story generation tasks demonstrate that PEFT significantly enhances the diversity of LLM outputs, as evidenced by lower similarity between candidate responses. Since PEFT emphasizes semantic diversity over lexical diversity, it can also notably reduce demographic bias in dialogue systems. The implementations and datasets are available in our repository: https://github.com/mailong25/peft_diversity
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。