通过精准定位神经元，让模型在混乱信息中学会拒答和提取关键证据。
Where Does Robustness Live? Neuron-Guided Adaptation for Retrieval-Augmented Language Models
- 按上下文激活情况定位关键神经元，实现细粒度模型调优
- 在多个问答数据集上显著优于基线方法，提升稳定性和准确性
- 适合需要高鲁棒性的知识密集型任务，如智能客服、医疗问答
检索增强语言模型（RALMs）在知识密集型任务中表现优异，但在检索内容噪声大或无关时仍易出错。鲁棒性依赖两种能力：无信息时拒绝生成，有信息时从中提取。现有方法既未分别训练这两种能力，也仅以粗粒度方式调整模型层或模块，忽略了仅少数神经元对特定输入强烈激活的事实。本文提出NeuRIT——一种基于定位优先视角的神经元引导鲁棒指令微调框架。NeuRIT挖掘与相关及无关上下文处理相关的上下文感知神经元，并将其作为锚点，选择性地调整这些神经元组及其集中所在的层。随后进行两阶段指令微调，分别教授‘无信息则不生成’和‘有信息则提取’的行为。NeuRIT在多个问答基准和不同生成器骨干上持续超越强基线方法。代码已开源：https://github.com/HYU-ARK-Lab/NeuRIT。
原文摘要 · Abstract (English)
Retrieval-Augmented Language Models (RALMs) have shown strong potential in knowledge-intensive tasks, yet they remain vulnerable when retrieved contexts are noisy or irrelevant. Robustness against such contexts requires two distinct capabilities: abstention when contexts are uninformative, and selective extraction when relevant evidence is buried in noise. Yet existing methods face two key limitations: they do not train separately for these two capabilities, and they adapt the model at a coarse layer- or module-level granularity, overlooking that only a small subset of neurons is strongly activated for a given input. We propose NeuRIT, a Neuron-guided Robust Instruction-Tuning framework built on a localization-first perspective. NeuRIT mines context-aware neurons associated with relevant and irrelevant context processing, and uses them as anchors to selectively adapt both the identified neuron groups and the layers in which they concentrate. NeuRIT then performs two-stage instruction tuning that teaches complementary behaviors: suppress generation when there is nothing to extract, and extract relevant evidence when there is. NeuRIT consistently outperforms strong baselines across diverse QA benchmarks and generator backbones. Our code is available at https://github.com/HYU-ARK-Lab/NeuRIT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。