用信息论设计评分机制，让AI写作更原创且符合要求
Thinking Outside the (Gray) Box: A Context-Based Score for Assessing Value and Originality in Neural Text Generation
- 基于上下文设计新评分标准，平衡创意与准确
- 在诗歌和数学题生成中提升输出价值与独特性
- 适合需要高质量创意输出的AI系统优化
尽管大语言模型广泛用于创造性任务，其输出常缺乏多样性。常用方法如提高采样温度会牺牲质量。为应对这一权衡挑战，本文基于信息论提出一种上下文相关的评分机制，可定量评估生成内容的价值与原创性。该评分鼓励准确性与需求契合度，同时促进偏离训练分布的创造性表达。实验表明，该评分可作为强化学习中的奖励函数，用于微调大模型，在诗歌生成、数学问题求解等多样创意任务中显著提升生成结果的价值与原创性。
原文摘要 · Abstract (English)
Despite the increasing use of large language models for creative tasks, their outputs often lack diversity. Common solutions, such as sampling at higher temperatures, can compromise the quality of the results. Dealing with this trade-off is still an open challenge in designing AI systems for creativity. Drawing on information theory, we propose a context-based score to quantitatively evaluate value and originality. This score incentivizes accuracy and adherence to the request while fostering divergence from the learned distribution. We show that our score can be used as a reward in a reinforcement learning framework to fine-tune large language models for maximum performance. We validate our strategy through experiments considering a variety of creative tasks, such as poetry generation and math problem solving, demonstrating that it enhances the value and originality of the generated solutions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。