让大模型像专业作家一样思考，生成更连贯的长文本。
SuperWriter: Reflection-Driven Long-Form Generation with Large Language Models
- 引入分步思维规划与优化，模拟写作者的创作流程。
- 在多个评测中超越更大模型，人类与自动评价均领先。
- 适合需要高质量长文本生成的研究与应用
长文本生成仍是大语言模型的重大挑战，尤其在序列长度增加时难以保持连贯性、逻辑一致性和文本质量。为此，我们提出 SuperWriter-Agent——一个基于智能体的框架，将结构化思维规划与迭代优化阶段显式融入生成流程，引导模型采用更严谨、类人化的创作方式。基于该框架，我们构建了一个监督微调数据集，并训练出70亿参数的 SuperWriter-LM。进一步设计了分层直接偏好优化（DPO）方法，利用蒙特卡洛树搜索（MCTS）传播最终质量评估，逐阶段优化生成过程。在多种基准上的实证结果表明，SuperWriter-LM 在自动评价与人工评价中均达到当前最优性能，甚至超越更大规模基线模型。全面的消融实验验证了分层DPO的有效性，并证明结构化思维步骤对提升长文本生成质量至关重要。
原文摘要 · Abstract (English)
Long-form text generation remains a significant challenge for large language models (LLMs), particularly in maintaining coherence, ensuring logical consistency, and preserving text quality as sequence length increases. To address these limitations, we propose SuperWriter-Agent, an agent-based framework designed to enhance the quality and consistency of long-form text generation. SuperWriter-Agent introduces explicit structured thinking-through planning and refinement stages into the generation pipeline, guiding the model to follow a more deliberate and cognitively grounded process akin to that of a professional writer. Based on this framework, we construct a supervised fine-tuning dataset to train a 7B SuperWriter-LM. We further develop a hierarchical Direct Preference Optimization (DPO) procedure that uses Monte Carlo Tree Search (MCTS) to propagate final quality assessments and optimize each generation step accordingly. Empirical results across diverse benchmarks demonstrate that SuperWriter-LM achieves state-of-the-art performance, surpassing even larger-scale baseline models in both automatic evaluation and human evaluation. Furthermore, comprehensive ablation studies demonstrate the effectiveness of hierarchical DPO and underscore the value of incorporating structured thinking steps to improve the quality of long-form text generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。