用大模型反馈训练小模型，让摘要更准确简洁
Learning to Summarize from LLM-generated Feedback
- 用多维度大模型反馈指导摘要生成
- 小模型经训练后优于大模型的摘要质量
- 适合想提升摘要性能的研究者与开发者
由于大模型生成摘要常出现幻觉、关键信息遗漏和冗长等问题，本文探索利用大模型生成的多维度反馈来提升摘要质量，使其更符合人类对忠实性、完整性和简洁性的偏好。我们构建了 FeedSum 大规模数据集，包含跨多个领域的不同质量摘要及其多维度的模型反馈。实验表明，高质量、多维度、细粒度的反馈能显著提升摘要生成效果。对比了监督微调与直接偏好优化两种方法，并提出 SummLlama3-8b 模型，在生成人类偏好的摘要上表现优于几乎 10 倍大的 Llama3-70b-instruct。相关数据集与模型已开源。
原文摘要 · Abstract (English)
Developing effective text summarizers remains a challenge due to issues like hallucinations, key information omissions, and verbosity in LLM-generated summaries. This work explores using LLM-generated feedback to improve summary quality by aligning the summaries with human preferences for faithfulness, completeness, and conciseness. We introduce FeedSum, a large-scale dataset containing multi-dimensional LLM feedback on summaries of varying quality across diverse domains. Our experiments show how feedback quality, dimensionality, and granularity influence preference learning, revealing that high-quality, multi-dimensional, fine-grained feedback significantly improves summary generation. We also compare two methods for using this feedback: supervised fine-tuning and direct preference optimization. Finally, we introduce SummLlama3-8b, a model that outperforms the nearly 10x larger Llama3-70b-instruct in generating human-preferred summaries, demonstrating that smaller models can achieve superior performance with appropriate training. The full dataset and SummLlama3-8B model are available at https://huggingface.co/datasets/DISLab/FeedSum and https://huggingface.co/DISLab/SummLlama3-8B.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。