arXiv:2608.05813cs.AI2026-08

让大模型动态决定何时用用户背景,提升个性化生成效果。

Cautious Context Steering for Language Model Personalization

论文配图:Cautious Context Steering for Language Model Personalization
图 1 · 摘自论文原文
  • 用轻量适配器控制每步生成中用户上下文的影响强度。
  • 仅用一个数据集训练,在多个新用户和场景上表现更优。
  • 无需为每个用户微调,推理成本低于现有方法。

将语言模型个性化以匹配不同用户的偏好至关重要。现有方法通常为每位用户单独训练适配器或学习依赖用户的奖励模型,但受限于少量观测,存在数据稀疏和泛化能力差的问题。在上下文学习(ICL)和上下文引导(CoS)中,直接利用用户上下文并借助预训练模型能力实现个性化,但未动态调节上下文影响:ICL无控制,CoS固定系数且每步需两次前向传播。本文提出谨慎上下文引导(CCS),在冻结主干模型上添加轻量适配器,实时判断每步是否及如何受用户上下文影响。适配器从一个理想上下文条件模型中学习,当上下文无帮助时保留原始模型行为。仅在一个数据集上训练的单个CCS适配器,在域内及四个跨域个性化基准上均提升生成质量,展现出对新用户和新领域的强泛化能力。同时避免了逐用户微调和额外的上下文条件前向传播,显著降低推理开销。

原文摘要 · Abstract (English)

Personalizing language models (LMs) to individual user preferences is essential for aligning responses with diverse goals and backgrounds. Existing methods typically train a separate adapter for each user or learn a reward model whose scores depend on the user. Despite explicitly optimizing for each user, these methods must learn from limited observations and therefore suffer from data sparsity and poor generalization to unseen users and domains. In-context learning (ICL) and Context Steering (CoS) can instead provide more effective personalization by conditioning the base LM directly on user context and leveraging its pretrained capabilities without per-user training. Yet neither adapts the influence of that context across decoding steps: ICL leaves it uncontrolled, whereas CoS applies a fixed steering coefficient and requires two LM forward passes per step. We propose Cautious Context Steering (CCS), which adds a lightweight adapter to a frozen backbone LM to decide at each token whether and how strongly user context should affect generation. The adapter learns this behavior from an oracle context-conditioned LM and preserves the base LM when the context is not helpful. A single CCS adapter trained on only one dataset improves generation quality both in-domain and across four out-of-distribution personalization benchmarks, demonstrating robust generalization to new users and domains. CCS also avoids per-user fine-tuning and the additional context-conditioned forward pass required by CoS, substantially reducing inference cost.

模型个性化上下文引导轻量化适配推理效率

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。