arXiv:2505.00038cs.CL2025-05被引 6

用用户写作特点生成个性化回复,比传统方法更精准可靠。

HyPerAlign: Interpretable Personalized LLM Alignment via Hypothesis Generation

  • 基于用户少量文本推断其表达习惯与性格特征
  • 作者归属任务胜率超90%,对话帮助性提升70%
  • 适合需要高度定制化输出的场景,如个人助手

对齐算法通常通过聚合多样用户的偏好来调整大语言模型(LLM),但这些模型往往反映的是“平均用户”偏好。而实际使用中,用户在特定情境下有独特需求,亟需个性化控制。本文提出HyPerAlign,一种基于假设生成的可解释、高效个性化的LLM对齐方法。给定少数目标用户的文本样本,先推断其沟通策略、性格和写作风格,再结合这些假设与用户属性,引导模型生成定制化输出。在作者归属与协商对齐两个任务上进行实验,数据来自新闻、博客、邮件及越狱测试集。结果表明,相比主流偏好微调方法,该方法在不同用户群体和模型上均表现更优:作者归属胜率普遍超过90%;协商对齐的帮助性平均提升70%。整体上,该方法为个体用户提供了可解释且样本高效的个性化方案。

原文摘要 · Abstract (English)

Alignment algorithms are widely used to align large language models (LLMs) to human users based on preference annotations. Typically these (often divergent) preferences are aggregated over a diverse set of users, resulting in fine-tuned models that are aligned to the ``average-user'' preference. Nevertheless, current models are used by individual users in very specific contexts and situations, emphasizing the need for user-dependent preference control. In this work we address the problem of personalizing LLM outputs to their users. We aim to generate customized responses tailored to specific individuals instead of generic outputs that emulate the collective voices of diverse populations. We propose HyPerAlign, an interpretable and sample-efficient hypothesis-driven personalization approach for LLM models. Given few-shot examples written by a particular user, we first infer hypotheses about their communication strategies, personality, and writing style, then prompt LLM models with these hypotheses and user-specific attributes to generate customized outputs. We conduct experiments on two different personalization tasks, namely authorship attribution and deliberative alignment, with datasets from diverse domains (news articles, blog posts, emails, jailbreaking benchmarks). Results demonstrate the superiority of hypothesis-driven LLM personalization compared to preference-based fine-tuning methods. For authorship attribution, HyPerAlign generations have consistently high win-rates (commonly $> 90\%$) against state-of-the-art preference fine-tuning approaches across diverse user profiles and LLM models. For deliberative alignment, the helpfulness of LLM models is improved by up to $70\%$ on average. Overall, HyPerAlign represents an interpretable and sample-efficient strategy for the personalization of LLM models to individual users.

个性化可解释少样本语言模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。