让大模型学会专注关键特征,自动忽略干扰信息。
Focus On This, Not That! Steering LLMs with Adaptive Feature Specification
- 训练模型根据指定特征响应,忽略无关信息
- 提升任务鲁棒性,抑制社会偏见,适应新特征
- 适合需要可控、公平推理的场景
尽管指令微调(IT)在训练大语言模型(LLMs)方面取得成功,但这些模型常依赖训练数据中习得的虚假或偏见特征,导致行为偏离预期。现有方法虽可在推理时调整模型行为,但多为事后补救,未将控制机制内嵌于模型。本文提出焦点指令微调(FIT),训练模型依据特定特征生成回应,同时忽略其他特征,实现不同行为模式。在多个基准测试中,我们验证了FIT:(i) 能有效实现推理时行为引导;(ii) 通过增强核心任务信号、降低虚假线索影响,提升鲁棒性;(iii) 通过抑制人口属性减少社会偏见;(iv) 在分布外情形和未见焦点特征下仍具泛化能力。因此,FIT提供了一种轻量级、内在化的机制,使模型更鲁棒、公平且易于控制。
原文摘要 · Abstract (English)
Despite the success of Instruction Tuning (IT) in training large language models (LLMs), such models often leverage spurious or biased features learnt from their training data and can become misaligned, leading to undesired behaviours. While existing techniques can steer model behaviour at inference-time, they are often post-hoc and do not embed steering as an intrinsic model feature. In this work, we introduce Focus Instruction Tuning (FIT), which trains LLMs to condition their responses by focusing on specific features whilst ignoring others, leading to different behaviours based on what features are specified. Across diverse benchmarks, we demonstrate that FIT: (i) successfully steers behaviour at inference time; (ii) increases robustness by amplifying core task signals and down-weighting spurious cues; (iii) mitigates social bias by suppressing demographic attributes; and (iv) generalises under distribution shifts and to previously unseen focus features. FIT therefore offers a lightweight, intrinsic mechanism for building more robust, fair, and easily controllable LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。