用神经元激活模式精选指令数据,提升模型特定任务表现。
Task-Specific Data Selection for Instruction Tuning via Monosemantic Neuronal Activations
- 以模型内部神经元激活模式表示样本,更真实反映处理过程。
- 通过稀疏自编码器解耦多义激活,实现更精准的数据相似性度量。
- 在多种任务和模型上稳定提升性能,适合需高精度指令调优的场景。
指令微调能增强大语言模型对多样化人类指令的理解能力,但在特定目标任务上取得强性能仍具挑战性,关键瓶颈在于如何选择最相关数据以最大化任务表现。现有数据选择方法包括不稳定的基于影响的方法和更稳定的分布对齐方法,后者高度依赖样本表示质量。实践中,从浅层特征(如BM25)到神经嵌入(如BGE、LLM2Vec)的分布对齐方法,往往无法准确捕捉模型内部对样本的处理机制。为此,本文提出以模型为中心的策略:用样本在模型中的神经元激活模式表示其内部计算过程。然而,直接使用原始神经元激活会因神经元多义性(单个神经元响应多个无关概念)导致无关样本间出现虚假相似性。为此,我们采用稀疏自编码器将多义激活分解为稀疏且单一语义的表示,并设计专用相似性度量用于该空间,以更好识别任务相关数据。在多个指令数据集、模型、任务及选择比例下的综合实验表明,本方法在稳定性与任务特定性能上均持续优于现有基线。
原文摘要 · Abstract (English)
Instruction tuning improves the ability of large language models (LLMs) to follow diverse human instructions, but achieving strong performance on specific target tasks remains challenging. A critical bottleneck is selecting the most relevant data to maximize task-specific performance. Existing data selection approaches include unstable influence-based methods and more stable distribution alignment methods, the latter of which critically rely on the underlying sample representation. In practice, most distribution alignment methods, from shallow features (e.g., BM25) to neural embeddings (e.g., BGE, LLM2Vec), may fail to capture how the model internally processes samples. To bridge this gap, we adopt a model-centric strategy in which each sample is represented by its neuronal activation pattern in the model, directly reflecting internal computation. However, directly using raw neuron activations leads to spurious similarity between unrelated samples due to neuron polysemanticity, where a single neuron may respond to multiple, unrelated concepts. To address this, we employ sparse autoencoders to disentangle polysemantic activations into sparse, monosemantic representations, and introduce a dedicated similarity metric for this space to better identify task-relevant data. Comprehensive experiments across multiple instruction datasets, models, tasks, and selection ratios show that our approach consistently outperforms existing data selection baselines in both stability and task-specific performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。