让大模型精准关注用户指令重点，动态调整注意力。
Spotlight Your Instructions: Instruction-following with Dynamic Attention Steering
- 推理时动态调节模型对指令关键部分的注意力比例。
- 在多指令任务中显著提升指令遵循准确率，无性能损失。
- 适用于不同规模模型，无需预训练或复杂设置。
在实际应用中，用户常通过自然语言指令引导大语言模型完成各类任务，但这些指令往往复杂多变，而模型无法稳定关注关键信息。现有方法要么依赖静态指令，要么需大量离线分析或固定偏置。为此，本文提出一种推理时动态注意力引导方法，用户可指定提示中重要部分，系统实时调整模型对这些部分的关注程度，使模型感知的重要性与用户意图一致。相比以往方法，该方法不依赖预训练，能动态更新注意力分配，显著提升多指令场景下的指令遵循能力，并在不同规模模型间有效泛化，且无性能下降风险。
原文摘要 · Abstract (English)
In many real-world applications, users rely on natural language instructions to guide large language models (LLMs) across a wide range of tasks. These instructions are often complex, diverse, and subject to frequent change. However, LLMs do not always attend to these instructions reliably, and users lack simple mechanisms to emphasize their importance beyond modifying prompt wording or structure. To address this, we present an inference-time method that enables users to emphasize specific parts of their prompt by steering the model's attention toward them, aligning the model's perceived importance of different prompt tokens with user intent. Unlike prior approaches that are limited to static instructions, require significant offline profiling, or rely on fixed biases, we dynamically update the proportion of model attention given to the user-specified parts--ensuring improved instruction following without performance degradation. We demonstrate that our approach improves instruction following across a variety of tasks involving multiple instructions and generalizes across models of varying scales.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。