无需微调，用少量例子让大模型学会用户个人写作风格。
Tuning-Free Personalized Alignment via Trial-Error-Explain In-Context Learning
- 通过试错解释机制动态构建个性化提示，提升风格匹配度。
- 在邮件、文章等任务中，对齐效果超越91.5%的基线模型。
- 适合需要快速适配用户风格但无法微调的场景使用。
语言模型通常反映群体共性，输出通用内容，难以契合特定用户的写作风格。本文提出无需微调的试错解释上下文学习方法（TICL），仅需每位用户少于10个示例即可实现个性化生成。TICL通过试错解释过程迭代扩展上下文提示，引入模型生成的负例样本与解释，提供细粒度指导以贴近用户风格。在基于LLM作为裁判的成对比较中，TICL取得高达91.5%的胜率，优于现有最先进方法。在邮件、论文和新闻写作任务中，其性能显著超过其他无微调基线。词法与定性分析表明，负例与解释帮助模型更有效地学习风格上下文，缓解零样本输出中对结构化、正式表达的偏好。通过提前消耗推理计算构建用户专属上下文提示，TICL在测试时无需额外生成步骤，为个性化对齐提供了一种新颖而简洁的解决方案。
原文摘要 · Abstract (English)
Language models are aligned to the collective voice of many, resulting in generic outputs that do not align with specific users' styles. In this work, we present Trial-Error-Explain In-Context Learning (TICL), a tuning-free method that personalizes language models for text generation tasks with fewer than 10 examples per user. TICL iteratively expands an in-context learning prompt via a trial-error-explain process, adding model-generated negative samples and explanations that provide fine-grained guidance towards a specific user's style. TICL achieves favorable win rates on pairwise comparisons with LLM-as-a-judge up to 91.5% against the previous state-of-the-art and outperforms competitive tuning-free baselines for personalized alignment tasks of writing emails, essays and news articles. Both lexical and qualitative analyses show that the negative samples and explanations enable language models to learn stylistic context more effectively and overcome the bias towards structural and formal phrases observed in their zero-shot outputs. By front-loading inference compute to create a user-specific in-context learning prompt that does not require extra generation steps at test time, TICL presents a novel yet simple approach for personalized alignment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。