让大模型在小显存上跑得更快，不改模型只靠系统优化
OOM-Free Alpamayo via CPU-GPU Memory Swapping for Vision-Language-Action Models
- 按层调度显存使用，逐层加载减少峰值占用
- 计算与传输重叠，隐藏数据搬移延迟
- 智能决定哪些层常驻显存，提速3.55倍适合部署者
面向自动驾驶的端到端视觉-语言-动作（VLA）模型将感知、推理与控制统一于单一神经网络，虽表现优异，但需20-60GB GPU显存，远超主流12-16GB显卡容量。本文提出仅通过系统级优化、无需修改模型的框架，实现显存受限GPU上的高效推理。工作分三步：(1) 逐层调度降低显存需求粒度；(2) 流水式调度利用传输-计算重叠隐藏数据搬运时间；(3) 基于模块驻留收益分析的决策策略，消除流水无法隐藏的残余开销。进一步提出性能预测模型，单次采样即可准确预测最优配置（误差<1.3%）。应用于NVIDIA Alpamayo-R1-10B（21.52GB）在RTX 5070Ti（16GB）上，相比Accelerate卸载提速达3.55倍，且保持全BF16精度。
原文摘要 · Abstract (English)
End-to-end Vision-Language-Action (VLA) models for autonomous driving unify perception, reasoning, and control in a single neural network, achieving strong driving performance but requiring 20-60GB of GPU memory-far exceeding the 12-16GB available on commodity GPUs. We present a framework, which enables memory-efficient VLA inference on VRAM-constrained GPUs through system-level optimization alone, without model modification. Our work proceeds in three stages: (1) Sequential Demand Layering reduces VRAM usage from model-level to layer-level granularity; (2) Pipelined Demand Layering hides parameter transfer time within layer execution time via transfer--compute overlap; and (3) a GPU-Resident Layer Decision Policy, informed by per-module residency benefit analysis, eliminates the residual transfer overhead that pipelining cannot hide. We further propose a performance prediction model that determines the optimal configuration-both the number and placement of resident layers-from a single profiling run with less than 1.3% prediction error across all configurations. Applied to NVIDIA's Alpamayo-R1-10B (21.52GB) on an RTX 5070Ti (16GB), our work achieves up to 3.55x speedup over Accelerate offloading while maintaining full BF16 precision.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。