arXiv:2510.24469cs.CLcs.AI2025-10

通过迭代反馈提升大模型个性化生成效果,无需训练即可改进风格与主题一致性。

Iterative Critique-Refine Framework for Enhancing LLM Personalization

  • 采用生成-批评-修正的循环框架,基于用户档案提供结构化反馈。
  • 在多个数据集上相比PGraphRAG提升7%-13%的个性化评分,3-5轮优化后稳定增益。
  • 无需训练、适配任意模型,适合需要快速部署个性化生成的场景。

个性化文本生成不仅要求内容连贯,还需匹配目标用户的风格、语气和主题聚焦。现有检索增强方法如LaMP和PGraphRAG虽通过用户及邻居历史丰富档案,但仅止于生成,常导致语气、主题或风格漂移。本文提出PerFine,一种统一的、无需训练的迭代式批判-修正框架,通过基于档案的持续反馈增强个性化。每轮中,生成器根据检索到的档案生成草稿,同档条件下的批评者LMM则对语气、词汇、句式与主题性提供结构化反馈,生成器据此修订;引入新颖的淘汰策略保留更强草稿。进一步研究了Best-of-N与主题提取等推理阶段策略,平衡质量与效率。在Yelp、Goodreads和Amazon数据集上,PerFine持续优于PGraphRAG,GEval得分提升7%-13%,3-5轮修正后表现稳定,且随批评者规模增大仍具可扩展性。结果表明,事后、档案感知的反馈是无需训练且模型无关的个性化生成有效范式。

原文摘要 · Abstract (English)

Personalized text generation requires models not only to produce coherent text but also to align with a target user's style, tone, and topical focus. Existing retrieval-augmented approaches such as LaMP and PGraphRAG enrich profiles with user and neighbor histories, but they stop at generation and often yield outputs that drift in tone, topic, or style. We present PerFine, a unified, training-free critique-refine framework that enhances personalization through iterative, profile-grounded feedback. In each iteration, an LLM generator produces a draft conditioned on the retrieved profile, and a critic LLM - also conditioned on the same profile - provides structured feedback on tone, vocabulary, sentence structure, and topicality. The generator then revises, while a novel knockout strategy retains the stronger draft across iterations. We further study additional inference-time strategies such as Best-of-N and Topic Extraction to balance quality and efficiency. Across Yelp, Goodreads, and Amazon datasets, PerFine consistently improves personalization over PGraphRAG, with GEval gains of +7-13%, steady improvements over 3-5 refinement iterations, and scalability with increasing critic size. These results highlight that post-hoc, profile-aware feedback offers a powerful paradigm for personalized LLM generation that is both training-free and model-agnostic.

个性化生成迭代优化无训练大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。