用新方法让大模型更像经典作家写故事。
Capturing Classic Authorial Style in Long-Form Story Generation with GRPO Fine-Tuning
- 先训练风格判别器，再用它指导模型生成。
- 80亿参数模型平均风格得分0.893，优于基线。
- 适合想控制文本风格的研究者和创作者。
长篇故事生成中的作者风格评估与优化仍具挑战，因风格常依赖随意提示，且易与整体写作质量混淆。本文提出两阶段流程：首先，通过作者身份验证监督微调句子编码器，将相似度输出校准为[0,1]区间奖励；其次，使用该奖励在组相对策略优化（GRPO）框架下微调80亿参数的故事生成模型，无需直接偏好优化（DPO）所需的接受/拒绝标注。在马克·吐温、简·奥斯汀、查尔斯·狄更斯、托马斯·哈代四位目标作者上，该模型平均风格得分为0.893，优于开源基线。结果表明，基于作者验证的奖励建模在中等规模模型和训练预算下，可实现有效的长篇风格可控生成。
原文摘要 · Abstract (English)
Evaluating and optimising authorial style in long-form story generation remains challenging because style is often assessed with ad hoc prompting and is frequently conflated with overall writing quality. We propose a two-stage pipeline. First, we train a dedicated style-similarity judge by fine-tuning a sentence-transformer with authorship-verification supervision, and calibrate its similarity outputs into a bounded $[0,1]$ reward. Second, we use this judge as the primary reward in Group Relative Policy Optimization (GRPO) to fine-tune an 8B story generator for style-conditioned writing, avoiding the accept/reject supervision required by Direct Preference Optimization (DPO). Across four target authors (Mark Twain, Jane Austen, Charles Dickens, Thomas Hardy), the GRPO-trained 8B model achieves higher style scores than open-weight baselines, with an average style score of 0.893 across authors. These results suggest that AV-calibrated reward modelling provides a practical mechanism for controllable style transfer in long-form generation under a moderate model size and training budget.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。