arXiv:2607.08991cs.LGcs.CL2026-07

通过感知敏感度的阈值法与动态路由,提升大模型推理效率。

Sensitivity-Aware Thresholding and Token Routing for Activation Sparsification in Large Language Models

论文配图:Sensitivity-Aware Thresholding and Token Routing for Activation Sparsification in Large Language Models
图 1 · 摘自论文原文
  • 用局部敏感度代理替代百分位,自动校准每层阈值。
  • 动态按令牌选择计算路径,比固定修改更优。
  • 在保持精度前提下,显著提升推理速度,适合部署优化。

大型语言模型(LLM)高效推理需在不损失模型质量的前提下决定计算缩减位置。本文聚焦多层感知机(MLP)激活稀疏化与令牌级条件路由。提出敏感度感知阈值稀疏化方法(SATS),通过局部输出敏感度代理而非激活百分位来校准各层门控阈值,保留原有阈值稀疏机制但改进校准方式。进一步设计轻量级令牌路由框架,实现每个令牌动态选择基础路径或优化路径,而非对所有令牌统一应用修改。在多个近期开源大模型上评估,结果表明SATs在相同实际稀疏度下优于基准阈值稀疏化方法;令牌路由相较静态激活修改基线,实现更优的质量-吞吐权衡。整体表明,改进阈值校准与令牌路由可有效提升大模型的质量-吞吐平衡。

原文摘要 · Abstract (English)

Efficient inference in Large Language Models (LLMs) requires deciding where computation can be reduced while preserving model quality. We study this problem through multilayer perceptron (MLP) activation sparsification and token-level conditional routing. We first propose Sensitivity-Aware Thresholding for Sparsity (SATS), a threshold calibration method to choose layerwise gate thresholds using a local MLP output sensitivity proxy rather than calibrating thresholds directly from activation percentiles. While SATS retains the existing mechanism of sparsifying MLP activations by thresholding gate activations, it replaces percentile-based calibration with a sensitivity-aware selection rule. We then introduce a lightweight token routing framework that dynamically selects between a base path and a modified path on a per-token basis, rather than applying the modified computation uniformly to all tokens. We evaluate both methods on multiple recent open-weight LLMs. Our results show that SATS improves over the threshold-based sparsification baseline at matched actual sparsity and that token routing yields a more favorable quality-throughput trade-off than static activation modification baselines. Overall, our results suggest that improved threshold calibration and token routing can improve the quality-throughput trade-off in LLMs.

模型压缩推理优化稀疏化路由机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。