4-bit量化+动态提前退出，让大模型在边缘设备上实时导航
EdgeNav-QE: QLoRA Quantization and Dynamic Early Exit for LAM-based Navigation on Edge Devices
- 用4位量化压缩模型，搭配可提前退出的分支结构
- 推理延迟降低82.7%，内存减少66.7%，成功率仍达81.8%
- 适合对实时性要求高的边缘智能导航场景
大型行动模型（LAM）通过连接高层推理与底层控制，在自主导航中展现出巨大潜力。然而，由于内存限制和延迟要求，将这些参数量达数十亿的模型部署在边缘设备上仍面临重大挑战。本文提出EdgeNav-QE框架，融合4位量化低秩适配（QLoRA）与动态提前退出（DEE）机制，优化基于大模型的导航系统在边缘设备上的表现。通过将主干网络量化至4位精度，并合理设置提前退出分支，模型可在简单任务中提前终止推理，复杂任务则保留完整深度。在Habitat-Sim环境与Matterport3D数据集上，使用OpenVLA-7B作为主干模型的实验表明，相比全精度基线，EdgeNav-QE将推理延迟降低82.7%，内存占用减少66.7%，同时保持81.8%的导航成功率。此外，其延迟性能优于当前最优静态提前退出方法17.9%，验证了面向内容的自适应计算在安全关键应用中的优越性。
原文摘要 · Abstract (English)
Large Action Models (LAMs) have shown immense potential in autonomous navigation by bridging high-level reasoning with low-level control. However, deploying these multi-billion parameter models on edge devices remains a significant challenge due to memory constraints and latency requirements. In this paper, we propose EdgeNav-QE, a novel framework that integrates Quantized Low-Rank Adaptation (QLoRA) with a dynamic early-exit (DEE) mechanism to optimize LAMs for real-time edge navigation. By quantizing the backbone to 4-bit precision and strategically placing early-exit branches, we enable the model to terminate inference early for simple navigation tasks while retaining full depth for complex decision-making. Experimental results on the Habitat-Sim environment with Matterport3D dataset using OpenVLA-7B backbone, demonstrate that EdgeNav-QE reduces inference latency by 82.7% and memory footprint by 66.7% compared to full-precision baselines, while maintaining 81.8% navigation success rate. Furthermore, it outperforms state-of-the-art static early-exit method by 17.9% in latency, demonstrating the superiority of content-aware adaptive computation for safety-critical applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。