用学术评审数据微调大模型，显著提升长文本理解能力。
Abstract2Appendix: Academic Reviews Enhance LLM Long-Context Capabilities
- 用高质量学术评审数据，通过DPO方法微调模型，效果优于传统SFT。
- 仅用2000样本，模型在Qasper上提升2.6%，比phi-3高4.04分。
- 人类评审胜过顶级模型（如GPT-4o），适合追求推理与长程理解的研究者。
大语言模型在各类任务中表现优异，但在处理长上下文阅读方面仍存挑战。本研究探索利用高质量学术同行评审数据微调大模型，以增强其长文本理解能力。对比直接偏好优化（DPO）与监督微调（SFT）方法，结果表明DPO在性能和数据效率上均更优。实验显示，微调后模型在Qasper基准上较phi-3提升4.04分，且仅使用2000个样本即实现2.6%的性能增长。尽管受限于数据规模与处理成本，研究凸显了DPO与高质量数据在提升模型表现上的潜力。零样本测试进一步表明，经过聚合的人类评审远优于大模型生成回复，即便对GPT-4o也如此。这说明人类评审蕴含丰富信息、推理与长上下文检索能力，当前最先进模型仍未完全掌握。该发现强调了利用人类评审推动领域发展的巨大价值。
原文摘要 · Abstract (English)
Large language models (LLMs) have shown remarkable performance across various tasks, yet their ability to handle long-context reading remains challenging. This study explores the effectiveness of leveraging high-quality academic peer review data for fine-tuning LLMs to enhance their long-context capabilities. We compare the Direct Preference Optimization (DPO) method with the Supervised Fine-Tuning (SFT) method, demonstrating DPO's superiority and data efficiency. Our experiments show that the fine-tuned model achieves a 4.04-point improvement over phi-3 and a 2.6\% increase on the Qasper benchmark using only 2000 samples. Despite facing limitations in data scale and processing costs, this study underscores the potential of DPO and high-quality data in advancing LLM performance. Additionally, the zero-shot benchmark results indicate that aggregated high-quality human reviews are overwhelmingly preferred over LLM-generated responses, even for the most capable models like GPT-4o. This suggests that high-quality human reviews are extremely rich in information, reasoning, and long-context retrieval, capabilities that even the most advanced models have not fully captured. These findings highlight the high utility of leveraging human reviews to further advance the field.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。