用动态干扰向量实现大模型快速个性化,高效适应用户变化
Fints: Efficient Inference-Time Personalization for LLMs with Fine-Grained Instance-Tailored Steering
- 通过注入样本级干扰向量,在推理时动态调整模型行为
- 在数据稀疏和分布突变场景下仍保持高性能
- 可无缝集成到现有个性化方法中,适合实时交互场景
大型语言模型的快速发展催生了对个性化技术的迫切需求。尽管基于上下文学习的非参数方法存在局限,当前参数化适配方法如个性化高效微调与奖励建模也面临动态用户模式适应性差、数据效率低的问题。为此,我们提出一种细粒度实例定制化引导框架,从用户数据动态生成样本级干扰向量,并注入模型前向传播以实现个性化。该方法包含两个关键技术:细粒度引导组件,通过捕获注意力与MLP层激活信号捕捉细微特征;输入感知聚合模块,将这些信号融合为上下文相关的增强项。实验表明,该方法在短文本到长文本生成、网页函数调用等多场景下均表现优异,显著提升快速变化环境中的个性化性能,且对不同交互模式与上下文长度具有强鲁棒性。实现代码已开源。
原文摘要 · Abstract (English)
The rapid evolution of large language models (LLMs) has intensified the demand for effective personalization techniques that can adapt model behavior to individual user preferences. Despite the non-parametric methods utilizing the in-context learning ability of LLMs, recent parametric adaptation methods, including personalized parameter-efficient fine-tuning and reward modeling emerge. However, these methods face limitations in handling dynamic user patterns and high data sparsity scenarios, due to low adaptability and data efficiency. To address these challenges, we propose a fine-grained and instance-tailored steering framework that dynamically generates sample-level interference vectors from user data and injects them into the model's forward pass for personalized adaptation. Our approach introduces two key technical innovations: a fine-grained steering component that captures nuanced signals by hooking activations from attention and MLP layers, and an input-aware aggregation module that synthesizes these signals into contextually relevant enhancements. The method demonstrates high flexibility and data efficiency, excelling in fast-changing distribution and high data sparsity scenarios. In addition, the proposed method is orthogonal to existing methods and operates as a plug-in component compatible with different personalization techniques. Extensive experiments across diverse scenarios--including short-to-long text generation, and web function calling--validate the effectiveness and compatibility of our approach. Results show that our method significantly enhances personalization performance in fast-shifting environments while maintaining robustness across varying interaction modes and context lengths. Implementation is available at https://github.com/KounianhuaDu/Fints.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。