通过融合全局与局部信号，提升大模型推理时稀疏化的准确性和稳定性。
GLASS: Global-Local Aggregation for Inference-time Sparsification of LLMs
- 融合输入提示激活与模型内在优先级，动态选择关键神经元。
- 短提示长生成场景下，困惑度降低45.10%，KL散度减少25.73%。
- 无需训练，适配多种开源大模型，适合资源受限设备部署。
推理时稀疏化是将大语言模型部署于资源受限设备的有前景路径，但现有无训练方法通常仅基于输入提示估计前馈网络（FFN）神经元重要性。我们发现，仅依赖提示的信号往往不可靠，尤其在短提示和长文本生成场景中，导致掩码不准确、生成质量下降。为此，提出GLASS——一种即插即用的无训练框架，通过聚合两种互补的神经元关键性视角：局部提示特异性激活与全局模型固有先验，稳定动态FFN剪枝。GLASS利用秩聚合融合全局与局部信号，在提示较短时仍能生成稳健的关键神经元选择。我们将GLASS解释为基于排列的概率模型下的最大后验共识排序，为加权秩聚合规则提供了理论基础。将其应用于多种开源大语言模型，在短提示、长生成场景中显著优于先前无训练基线，困惑度最高降低45.10%，KL散度降低25.73%，同时实现显著的本地解码加速。
原文摘要 · Abstract (English)
Inference-time sparsification is a promising path to deploy large language models (LLMs) on resource-constrained devices, yet existing training-free methods typically estimate feedforward network (FFN) neuron importance from the input prompt alone. We show this prompt-only signal is often unreliable, especially for short prompts and long-form decoding, leading to inaccurate masks and degraded generation fidelity. We propose GLASS, a plug-and-play, training-free framework that stabilizes dynamic FFN pruning by aggregating two complementary views of neuron criticality: local prompt-specific activations and a global model-intrinsic prior. GLASS fuses global and local signals via rank aggregation, yielding robust critical-neuron selection even when the prompt is short. We interpret GLASS as the maximum-a-posteriori consensus ranking under a permutation-based probabilistic model, providing a principled foundation for its weighted rank-aggregation rule. We apply GLASS to a diverse set of open-source LLMs, and show that it yields substantial improvements over prior training-free baselines in the challenging short-prompt, long-generation scenarios, achieving up to 45.10% lower perplexity and 25.73% lower KL divergence, while delivering significant on-device decoding speedup.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。