用盲审反馈机制提升大模型的创意写作能力。
LLM Review: Enhancing Creative Writing via Blind Peer Review Feedback
- 引入盲审式互评，独立修改保持创意多样性。
- 在科幻写作数据集上显著优于多智能体基线。
- 小模型经此框架可超越大模型，适合资源有限场景。
大型语言模型在创意生成中表现不佳，而通过交互提升推理的多智能体框架反而导致内容趋同。本文提出LLM Review，一种受同行评审启发的框架：智能体间交换针对性反馈并独立修订，保留多样化的创作路径。为实现严谨评估，我们构建SciFi-100数据集，采用统一框架结合大模型评分、人工标注和基于规则的新颖性指标。实验表明，LLM Review持续优于多智能体基线，且小型模型经此框架可超越大型单智能体模型，表明互动结构可替代模型规模。
原文摘要 · Abstract (English)
Large Language Models (LLMs) often struggle with creative generation, and multi-agent frameworks that improve reasoning through interaction can paradoxically hinder creativity by inducing content homogenization. We introduce LLM Review, a peer-review-inspired framework implementing Blind Peer Review: agents exchange targeted feedback while revising independently, preserving divergent creative trajectories. To enable rigorous evaluation, we propose SciFi-100, a science fiction writing dataset with a unified framework combining LLM-as-a-judge scoring, human annotation, and rule-based novelty metrics. Experiments demonstrate that LLM Review consistently outperforms multi-agent baselines, and smaller models with our framework can surpass larger single-agent models, suggesting interaction structure may substitute for model scale.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。