将机器人视觉编码器迁至NPU可省电,但会增加训练时间并轻微降低性能。
NPU Offloading of a Frozen Visual Encoder for Robot Policy Training

- 用NPU处理冻结的视觉编码器,与GPU协同异步训练。
- 最多省电27.9%,但单步训练时间增加37.7%。
- 适合注重能效、可容忍小幅性能下降的机器人训练场景。
当为新任务或数据集训练机器人策略时,可冻结视觉编码器仅训练动作生成模块,降低训练成本。冻结虽移除了反向传播,但正向计算仍需每步执行,消耗大量GPU算力。本文探索将此计算迁移至低功耗AI加速器如NPU,以降低总能耗,同时评估对策略性能的影响。构建了基于GPU与NPU的异步训练流水线:在Mobilint Aries2 NPU上以A8W8 INT8运行冻结的视觉编码器(含ResNet18和4个Transformer层),而FP32动作专家在NVIDIA GeForce RTX 5060 Ti GPU上训练。对比了纯GPU基线与四种逐步扩展至1~4层编码器的NPU卸载条件(L1-L4),每种训练30,000步,三组随机种子。测量各条件下板级功耗:相比基线,L1(卸载ResNet18+第一层)节能17.1%,L4(卸载全部)节能27.9%;但单步训练时间分别增加15.2%和37.7%,峰值GPU内存减少19.8%~20.7%。15个策略均在相同300个环境种子下测试,共4,500次模拟回放。纯GPU策略成功率为93.33%,NPU条件为91.44%~92.89%,性能下降0.44~1.89个百分点。结果表明,NPU卸载可显著降低训练能耗,但代价是延长训练时间并小幅降低策略成功率。
原文摘要 · Abstract (English)
When a robot policy is trained for a new task or dataset, its visual encoder can be frozen and only its action generation module trained, reducing training cost. Freezing removes the encoder's backward pass, but its forward pass must still run at every training step because the input images change, so it keeps consuming GPU compute. We therefore ask whether moving this computation to a low power AI accelerator such as an NPU can reduce total energy despite the added data transfer and longer training time, and how it affects policy performance. We built an asynchronous training pipeline that uses both a GPU and an NPU for the AR-Actor specialist. The frozen visual encoder runs in A8W8 INT8 on a Mobilint Aries2 NPU, while the FP32 action expert is trained on an NVIDIA GeForce RTX 5060 Ti GPU. We compared a GPU-only baseline with four conditions, L1 to L4, which gradually extend NPU offloading from one to four Transformer encoder layers. Each condition was trained for 30,000 steps with three random seeds. We measured GPU board power for the GPU-only condition and combined GPU and NPU board power for the NPU conditions. Energy per sample decreased by 17.1% in L1, which offloaded ResNet18 and the first encoder layer, and by 27.9% in L4, which offloaded ResNet18 and all four encoder layers. In contrast, training time per sample increased by 15.2% in L1 and 37.7% in L4, and peak allocated GPU memory decreased by 19.8 to 20.7%. The 15 resulting policies were each evaluated with the same 300 environment seeds, for a total of 4,500 simulator rollouts. The combined success rate was 93.33% for GPU-only and 91.44 to 92.89% for the NPU conditions. These results show that NPU offloading of a frozen visual encoder can reduce training energy, but it increases training time and lowers policy success rate by 0.44 to 1.89 percentage points compared with GPU-only training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。