arXiv:2606.04095cs.CLcs.AI2026-06中稿 · EMNLP被引 1

小模型也能写出高质量长篇故事,靠的是好评委和人类参考模板。

POLARIS: Guiding Small Models to Write Long Stories

论文配图:POLARIS: Guiding Small Models to Write Long Stories
图 1 · 摘自论文原文
  • 用大模型当评委打分,结合人类写的故事作锚点引导训练。
  • 90亿参数模型生成3倍于训练长度的长故事,质量仍稳定。
  • 适合想提升小模型创作能力的研究者和开发者。

小规模开源模型在长文本创意写作上表现不佳:生成内容常达不到要求长度,或随长度增加质量明显下降,尤其相较于前沿模型。我们提出POLARIS(基于大模型评分与锚定参考注入的策略优化),一种低计算量的GRPO训练方法,包含两个关键设计:使用具备结构化评分标准的前沿大模型作为在线奖励信号,以及人类参考注入(HRI),将教师强制生成的人类写作文本作为每个训练组内的高奖励锚点。在约1400个来自100部短篇小说集和4个A100 GPU上,对Qwen3.5-9B应用该训练方案,得到POLARIS-9B。在五个涵盖分布内与分布外提示及评分标准的基准测试中,POLARIS-9B在遵循长度指令方面优于基线模型,且性能可媲美更大规模的开源模型。盲评结果表明,其优于原版Qwen3.5-9B,与Qwen3.5-27B相当。尽管训练仅覆盖最长4000词的故事,其在生成长达1.2万词(为训练上限3倍)故事时仍保持良好质量,而多数开源模型在此条件下会出现质量或长度控制严重退化。更广泛地,我们的结果表明,长度泛化是评估创意写作模型的重要压力测试,有助于区分原本表现接近的模型。

原文摘要 · Abstract (English)

Small open-weight models struggle at long-form creative writing: their generated stories either fall far short of the requested length, or their quality significantly degrades as length increases, especially when compared to frontier models. We present POLARIS (Policy Optimization with LLM-as-a-judge rewards and Anchored-Reference Injection for Storywriting), a lower-compute GRPO recipe with two key ingredients: a frontier LLM judge with a structured Story Quality rubric as the online reward, and human-reference injection (HRI), where a teacher-forced human-written story serves as a high-reward anchor within each GRPO group. By applying our training recipe to Qwen3.5-9B, using a dataset of approximately 1.4K prompt-story pairs derived from 100 short-story anthologies and 4 A100 GPUs, we obtain POLARIS-9B. Across five benchmarks spanning in-distribution and out-of-distribution prompts and rubrics, POLARIS-9B is competitive with much larger open-weight models while following length instructions more closely. A blinded human evaluation confirms that POLARIS-9B is preferred to the base Qwen3.5-9B and on par with Qwen3.5-27B. Despite training only on stories up to 4k words, POLARIS-9B preserves quality on prompts requesting stories up to 3 times the training length, a regime where most open-weight models degrade substantially in quality, length adherence, or both. More broadly, our results suggest that length generalization is a meaningful stress test for creative-writing models and a useful lens for distinguishing otherwise close models.

长文本生成小模型优化生成质量提示控制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。