根据词元复杂度动态分块，实现低秩微调的内存节省与推理加速。
ChunkWise LoRA: Adaptive Sequence Partitioning for Memory-Efficient Low-Rank Adaptation and Accelerated LLM Inference
- 按词元复杂度动态分块，每块配置不同低秩参数
- 相比基线方法降低34%延迟，减少38%内存占用
- 兼容现有架构，适合实际部署的高效微调场景
低秩微调（LoRA）虽能高效微调大语言模型，但传统方法对所有输入词元采用固定秩配置，忽略其复杂度差异。本文提出ChunkWise LoRA，根据词元复杂度动态划分序列成可变长度块，并为每块分配定制化的低秩配置。系统引入运行时调度器，估计词元难度，执行自适应分块，并通过秩阶梯机制选择每块的秩和缩放系数。为保证输出一致性，设计边界安全组合模块并集成策略驱动的键值缓存策略。在Wikitext-103和SQuAD等基准数据集上的实验表明，该方法相较基线LoRA可实现最高34%的延迟降低和38%的内存减少，同时保持或提升BLEU、EM和困惑度等任务性能指标。框架完全兼容现有Transformer架构与推理系统，为参数高效大模型的实际部署提供可行方案。
原文摘要 · Abstract (English)
Recent advances in low-rank adaptation (LoRA) have enabled efficient fine-tuning of large language models (LLMs) with minimal additional parameters. However, existing LoRA methods apply static rank configurations uniformly across all input tokens, ignoring variation in token complexity and computational requirements. In this work, we propose ChunkWise LoRA, a dynamic and adaptive approach that partitions sequences into variable-length chunks based on token complexity and assigns each chunk a tailored low-rank configuration. Our system introduces a runtime scheduler that estimates token difficulty, performs adaptive chunking, and selects per-chunk LoRA rank and scaling using a rank-ladder mechanism. To preserve output consistency, we further introduce a boundary-safe composition module and integrate policy-driven KV-cache strategies. Experiments on benchmark datasets such as Wikitext-103 and SQuAD demonstrate that ChunkWise LoRA achieves up to 34\% lower latency and 38% memory reduction compared to baseline LoRA, while maintaining or improving task performance metrics like BLEU, EM, and perplexity. The proposed framework remains fully compatible with existing transformer architectures and inference frameworks, providing a practical solution for real-world deployment of parameter-efficient LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。