AngelSlim整合多种大模型压缩技术,提升推理效率与部署便捷性。
AngelSlim: A more accessible, comprehensive, and efficient toolkit for large model compression
- 集成量化、推测解码、令牌剪枝等前沿算法,实现统一压缩流程。
- 支持FP8/INT8 PTQ及2比特模型,实现1.8~2.0倍吞吐提升。
- 适合关注模型轻量化与工业部署的研究者与工程师。
本技术报告介绍由腾讯混元团队开发的AngelSlim,一个全面且多功能的大模型压缩工具包。通过整合量化、推测解码、令牌剪枝与知识蒸馏等前沿算法,AngelSlim提供从压缩到工业级部署的统一流程。为实现高效加速,工具包集成最先进的FP8与INT8后训练量化(PTQ)算法,并在超低比特领域取得突破,推出首个可工业应用的2比特大模型HY-1.8B-int2。除量化外,提出适配多模态架构与现代推理引擎的训练对齐推测解码框架,实现1.8x~2.0x吞吐增益且不牺牲输出正确性。此外,开发无需训练的稀疏注意力框架,通过静态模式与动态令牌选择分离稀疏核,显著降低长上下文场景下的首令牌延迟(TTFT)。针对多模态模型,引入专用剪枝策略:基于最大边际相关性的IDPruner优化视觉令牌,Samp实现自适应音频令牌合并与剪枝。通过底层实现融合多种压缩策略,AngelSlim支持算法研究与工具化部署。
原文摘要 · Abstract (English)
This technical report introduces AngelSlim, a comprehensive and versatile toolkit for large model compression developed by the Tencent Hunyuan team. By consolidating cutting-edge algorithms, including quantization, speculative decoding, token pruning, and distillation. AngelSlim provides a unified pipeline that streamlines the transition from model compression to industrial-scale deployment. To facilitate efficient acceleration, we integrate state-of-the-art FP8 and INT8 Post-Training Quantization (PTQ) algorithms alongside pioneering research in ultra-low-bit regimes, featuring HY-1.8B-int2 as the first industrially viable 2-bit large model. Beyond quantization, we propose a training-aligned speculative decoding framework compatible with multimodal architectures and modern inference engines, achieving 1.8x to 2.0x throughput gains without compromising output correctness. Furthermore, we develop a training-free sparse attention framework that reduces Time-to-First-Token (TTFT) in long-context scenarios by decoupling sparse kernels from model architectures through a hybrid of static patterns and dynamic token selection. For multimodal models, AngelSlim incorporates specialized pruning strategies, namely IDPruner for optimizing vision tokens via Maximal Marginal Relevance and Samp for adaptive audio token merging and pruning. By integrating these compression strategies from low-level implementations, AngelSlim enables algorithm-focused research and tool-assisted deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。