压缩大模型时维度不匹配会拖慢推理速度，新方法可显著提升效率。
Why Smaller Is Slower? Dimensional Misalignment in Compressed LLMs
- 通过优化压缩后张量维度，使其适配GPU执行架构
- 在减少15%参数的同时实现1.5倍加速，且精度不变
- 适合关注模型部署性能的工程师与研究者
后训练压缩虽能减少大模型参数量，但常导致张量维度不规则，进而降低GPU性能——我们称之为‘维度不匹配’。本文从框架、库到硬件三个层面展开全栈分析，发现推理变慢的根本原因在于压缩后的维度不适应GPU执行栈。例如，用激活感知奇异值分解（ASVD）压缩Llama-3-8B，参数减少15%，但95%的维度仍不匹配，运行速度未提升。为此提出GAC（GPU对齐压缩）：一种新压缩范式，可封装任意降维压缩器，并在相同参数预算下，通过多选背包优化重选硬件友好的维度。在Llama-3-8B上结合ASVD和LLM-Pruner测试，实现100%维度对齐，恢复最高达1.5倍的加速，同时保持模型质量。
原文摘要 · Abstract (English)
Post-training compression reduces LLM parameter counts but often produces irregular tensor dimensions that degrade GPU performance -- a phenomenon we call \emph{dimensional misalignment}. We present a full-stack analysis tracing root causes at three levels: framework, library, and hardware. The key insight is that model inference becomes slower because the resulting dimensions are unfriendly with the GPU execution stack. For example, compressing Llama-3-8B with activation-aware singular value decomposition (ASVD) has 15\% fewer parameters yet runs no faster than the uncompressed baseline, because 95\% of its dimensions are misaligned. We propose \textbf{GAC} (GPU-Aligned Compression), a new compression paradigm that wraps any dimension-reducing compressor and re-selects hardware-aligned dimensions via multi-choice knapsack optimization under the same parameter budget. We evaluate GAC on Llama-3-8B with ASVD and LLM-Pruner, achieving 100\% alignment and recovering up to 1.5$\times$ speedup while preserving model quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。