arXiv:2512.15973cs.LGcs.CL2025-12

动态调整注意力秩,让大模型在长文本中更快更省资源。

Dynamic Rank Reinforcement Learning for Adaptive Low-Rank Multi-Head Self Attention in Large Language Models

  • 用强化学习实时决定每层注意力的秩,兼顾精度与速度。
  • 长序列下减少40%以上算力,准确率接近全秩模型。
  • 适合部署在资源受限设备上的大模型推理优化。

动态秩强化学习(DR-RL)依赖静态秩假设,限制了其在多样化语言场景中的灵活性。本文方法根据实时序列动态、层间敏感性及硬件约束动态调节注意力秩。核心创新在于构建深度强化学习代理,将秩选择建模为序列策略优化问题,严格平衡注意力保真度与计算延迟。为保障推理稳定性,推导并应用在线矩阵扰动界,实现增量秩更新而无需全分解的高昂开销。此外,轻量级Transformer策略网络与批处理奇异值分解(SVD)操作的结合,确保了现代架构下的可扩展部署。大量实验表明,DR-RL在长序列场景(L > 4096)下可降低超过40%的浮点运算量(FLOPs),同时保持下游任务准确率与全秩注意力统计等效。除标准语言建模基准外,我们在GLUE基准上验证了其实用性:在SST-2情感分析任务中达到92.78%准确率,与全秩基线持平,并显著优于Performer和Nyströmformer等静态低秩方法。

原文摘要 · Abstract (English)

Dynamic Rank Reinforcement Learning (DR-RL) approximations rely on static rank assumptions, limiting their flexibility across diverse linguistic contexts. Our method dynamically modulates ranks based on real-time sequence dynamics, layer-specific sensitivities, and hardware constraints. The core innovation is a deep reinforcement learning agent that formulates rank selection as a sequential policy optimization problem, strictly balancing attention fidelity against computational latency. To ensure stability during inference, we derive and employ online matrix perturbation bounds, enabling incremental rank updates without the prohibitive cost of full decomposition. Furthermore, the integration of a lightweight Transformer-based policy network and batched Singular Value Decomposition (SVD) operations ensures scalable deployment on modern architectures. Extensive experiments demonstrate that DR-RL significantly reduces Floating Point Operations (FLOPs) by over 40% in long-sequence regimes (L > 4096) while maintaining downstream accuracy statistically equivalent to full-rank attention. Beyond standard language modeling benchmarks, we validate the real-world applicability of DR-RL on the GLUE benchmark. Specifically, our method achieves 92.78% accuracy on the SST-2 sentiment analysis task, matching the performance of full-rank baselines and outperforming static low-rank methods, such as Performer and Nyströmformer, by a significant margin.

大模型压缩注意力机制强化学习高效推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。