推理模型的内存优化需按规模调整，小模型重参数，大模型重生成长度。
Not All Bits Are Equal: Scale-Dependent Memory Optimization Strategies for Reasoning Models
- 根据模型规模决定内存分配：小模型多存权重，大模型多存生成历史
- 有效参数量低于8比特40亿时，增加权重比延长生成更保准确率
- 为推理模型提供分规模的部署策略，颠覆传统统一量化思路
尽管4比特量化已成为非推理模型和零样本任务中的内存最优选择，但我们在AIME25和GPQA-Diamond上对1700个推理场景的系统实验发现，该通用方案在推理模型中失效——关键在于键值缓存（KV cache）而非模型大小主导内存占用。存在一个规模阈值：当模型有效参数量低于8比特40亿时，将内存用于更多权重可提升准确率；而更大模型则通过延长生成长度实现更高准确率。此阈值还决定了并行扩展何时内存高效，以及是否应采用缓存剔除而非量化。结果表明，大语言模型的内存优化不能一刀切，应针对不同规模制定策略：小模型优先提升模型容量，大模型则最大化测试时计算能力。这揭示了推理模型部署所需的优化策略与非推理模型根本不同。
原文摘要 · Abstract (English)
While 4-bit quantization has emerged as a memory-optimal choice for non-reasoning models and zero-shot tasks across scales, we show that this universal prescription fails for reasoning models, where the KV cache rather than model size can dominate memory. Through systematic experiments across 1,700 inference scenarios on AIME25 and GPQA-Diamond, we find a scale-dependent trade-off: models with an effective size below 8-bit 4B parameters achieve better accuracy by allocating memory to more weights rather than longer generation, while larger models achieve better accuracy by allocating memory to longer generations. This scale threshold also determines when parallel scaling becomes memory-efficient and whether KV cache eviction outperforms KV quantization. Our findings show that memory optimization for LLMs cannot be scale-agnostic, while providing principled guidelines: for small reasoning models, prioritize model capacity over test-time compute, while for larger ones, maximize test-time compute. Our results suggest that optimizing reasoning models for deployment requires fundamentally different strategies from those established for non-reasoning models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。