小数据下精准匹配用户偏好，让大模型更懂你。
FaST: Feature-aware Sampling and Tuning for Personalized Preference Alignment with Limited Data
- 用数据自动提取高层特征，高效适配用户偏好
- 在仅少量标注数据下，性能优于现有方法
- 适用于个性化对话助手，尤其适合数据稀缺场景
基于大语言模型的对话助手机能常以统一方式部署，难以适应个体用户偏好。近年来，模型个性化——即根据特定用户偏好调整模型——成为弥合这一差距的重要方向。本文聚焦一个实际且具挑战性的场景：每位用户仅能收集少量偏好标注数据，该问题被定义为有限数据下的个性化偏好对齐（PPALLI）。为此，我们构建了两个数据集DnD和ELIP，并在此基础上评估多种对齐技术。同时提出一种名为FaST的高参数效率方法，通过自动挖掘数据中的高层特征，在小样本条件下实现最优性能。
原文摘要 · Abstract (English)
LLM-powered conversational assistants are often deployed in a one-size-fits-all manner, which fails to accommodate individual user preferences. Recently, LLM personalization -- tailoring models to align with specific user preferences -- has gained increasing attention as a way to bridge this gap. In this work, we specifically focus on a practical yet challenging setting where only a small set of preference annotations can be collected per user -- a problem we define as Personalized Preference Alignment with Limited Data (PPALLI). To support research in this area, we introduce two datasets -- DnD and ELIP -- and benchmark a variety of alignment techniques on them. We further propose FaST, a highly parameter-efficient approach that leverages high-level features automatically discovered from the data, achieving the best overall performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。