arXiv:2606.05868cs.CL2026-06

通过动态调整注意力结构,显著降低金融大模型推理内存占用,提升并发性能。

YouZhi: Towards High-Concurrency Financial LLMs via Adaptive GQA-to-MLA Transition

论文配图:YouZhi: Towards High-Concurrency Financial LLMs via Adaptive GQA-to-MLA Transition
图 1 · 摘自论文原文
  • 按层自适应切换分组查询到多查询注意力,优化缓存压缩效率。
  • 7B模型并发能力提升2.69倍,14B模型推理准确率提高7.0%。
  • 专为华为Ascend芯片设计,适合金融领域高并发部署场景。

大型语言模型推动金融领域创新,但高并发部署受限于键值缓存(KV cache)内存开销,导致成本上升、扩展性下降。为此,我们提出YouZhi-LLM,一种基于华为Ascend生态的高效金融大模型,采用完整的结构转换与训练流程。其核心是层自适应的GQA-to-MLA动态转换框架,按层灵活分配频次折叠大小,在最大化KV缓存压缩的同时最小化困惑度损失。为恢复表征能力并注入领域知识,训练流程融合通用知识蒸馏与金融特定监督微调。实验表明,该方法相比均匀基线可将困惑度损失降低最多35%。在Ascend NPUs上使用vLLM-Ascend评估,大规模缓存缩减直接转化为部署效率提升:相比基础模型,YouZhi-7B平均金融基准得分提升12.3%,最大并发量提高2.69倍;YouZhi-14B实现7.0%精度增益与2.43倍并发提升,确立了低成本、高吞吐金融推理的新范式。

原文摘要 · Abstract (English)

Large language models (LLMs) drive significant financial innovations, yet their high-concurrency deployment is severely bottlenecked by KV cache memory overhead, which inflates infrastructure costs and throttles scalability. To address this, we propose YouZhi-LLM, a highly efficient financial LLM empowered by a comprehensive structural transition and training pipeline natively built on the Huawei Ascend ecosystem. At its algorithmic core, YouZhi-LLM features a layer-adaptive GQA-to-MLA transition framework that dynamically assigns per-layer FreqFold sizes, maximizing KV-cache compression while minimizing perplexity degradation. To recover representation capacity and inject domain expertise, the Ascend-based training pipeline seamlessly integrates generalized knowledge distillation with financial-specific supervised fine-tuning. Evaluations demonstrate the superiority of this systematic approach, with the adaptive transition reducing perplexity degradation by up to 35% over uniform baselines. Crucially, when evaluated on Ascend NPUs via vLLM-Ascend, the massive KV-cache reduction translates directly into deployment efficiency. Compared to their respective base models, YouZhi-7B yields a 12.3% improvement in average financial benchmark score alongside a 2.69$\times$ increase in maximum concurrency; similarly, YouZhi-14B achieves a 7.0% accuracy gain and a 2.43$\times$ concurrency boost, establishing a new paradigm for cost-effective, high-throughput financial inference.

金融大模型注意力机制内存优化昇腾生态

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。