arXiv:2603.06745cs.LGcs.AI2026-03中稿 · ICLR被引 5

通过动态调节强化指令遵循,避免大模型生成质量下降。

Enhancing Instruction Following of LLMs via Activation Steering with Dynamic Rejection

  • 基于注意力敏感度分析,动态调整激活控制强度。
  • 在多个基准上提升指令遵循准确率最高达6.5%。
  • 适合需要精准指令响应的对话与内容生成场景。

尽管大型语言模型在指令微调方面取得进展,但仍难以遵循复杂用户指令。激活操纵技术虽能缓解此问题,但存在过强操纵导致任务准确率和文本质量下降的风险。为此,我们提出DIRECTER(动态拒绝引导操纵),一种无需额外数据集的操纵方法,通过缩放键值缓存动态调节操纵强度。DIRECTER将操纵与合理性引导解码循环结合,每一步比较操纵后输出分布与原始分布,若判断输出不合理则逐步减弱操纵强度。该强度调节由轻量级一次性注意力敏感度分析指导,按层排序对模型表征的影响。大量评估表明,DIRECTER显著提升多种基准上的指令遵循能力,相比基线最高提升6.5%准确率,且无生成质量或任务保真度的常见权衡。该动态、合理性引导的控制机制为缓解过操纵提供了通用方案,兼容现有基线。

原文摘要 · Abstract (English)

Large Language Models (LLMs), despite advances in instruction tuning, often fail to follow complex user instructions. Activation steering techniques aim to mitigate this by manipulating model internals, but have a potential risk of oversteering, where excessive emphasis on the instruction degrades task accuracy and overall text quality. To address this, we introduce DIRECTER (Dynamic rejection steering), a novel steering method that dynamically modulates steering strength by scaling the KV cache without extra dataset. DIRECTER couples steering with a plausibility-guided decoding loop, which adaptively adjusts steering strength at each step by comparing the steered output distribution to the original. If the steered output is deemed implausible, steering strength is progressively weakened. This strength modulation is guided by a lightweight, one-time attention sensitivity analysis that ranks layers by their influence on model representations. Extensive evaluations show that DIRECTER significantly enhances instruction-following capabilities across diverse benchmarks, improving accuracy by up to 6.5% over baselines without the common trade-offs in generation quality or task fidelity. The proposed dynamic, plausibility-guided control during activation steering further demonstrates its potential as a general mechanism for mitigating oversteering that is compatible with existing baselines.

指令遵循激活操纵大模型优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。