EdgeFM让视觉语言模型在边缘设备上更快更稳运行,打破厂商封闭生态。
EdgeFM: Efficient Edge Inference for Vision-Language Models
- 用智能代理自动优化底层计算内核,形成可复用的轻量模块。
- 在NVIDIA Orin上比TensorRT-Edge-LLM快1.49倍,支持多平台部署。
- 开源方案适合工业边缘场景,避免硬件锁定,提升跨平台兼容性。
视觉语言模型(VLMs)在边缘工业应用中展现出强大潜力,但其部署受限于确定性低延迟和资源受限下的稳定执行需求。现有框架或依赖臃肿的通用设计,或迫使开发者进入封闭的、硬件特定的专有生态,导致硬件锁定和跨平台适应性差。观察到现代AI代理可高效搜索并调优配置以生成高度优化的低级内核,我们提出EdgeFM——一种面向跨平台工业边缘部署的轻量级、代理驱动的VLM/LLM推理框架。EdgeFM通过移除非必要功能降低单请求延迟,并将代理调优的内核优化封装为可复用的模块化技能库。通过直接调用这些技能,而非等待闭源实现,有效弥合了长期由专有工具链主导的性能差距。该框架原生支持x86与NVIDIA Orin SoC等主流平台,首次实现国内Horizon Journey平台上的端到端VLA部署,显著提升跨平台可移植性。实验表明,多数情况下其推理性能优于传统厂商工具链,在NVIDIA Orin平台上相比TensorRT-Edge-LLM最高提速1.49倍。结果证明EdgeFM具备出色的端到端推理表现,为多样化的边缘工业场景提供开源、生产级解决方案。
原文摘要 · Abstract (English)
Vision-language models (VLMs) have demonstrated strong applicability in edge industrial applications, yet their deployment remains severely constrained by requirements for deterministic low latency and stable execution under resource limitations. Existing frameworks either rely on bloated general-purpose designs or force developers into opaque, hardware-specific closed-source ecosystems, leading to hardware lock-in limitation and poor cross-platform adaptability. Observing that modern AI agents can efficiently search and tune configurations to generate highly optimized low-level kernels for standard LLM operators, we propose EdgeFM, a lightweight, agent-driven VLM/LLM inference framework tailored for cross-platform industrial edge deployment. EdgeFM removes non-essential features to reduce single-request latency, and encapsulates agent-tuned kernel optimizations as a modular library of reusable skills. By allowing direct invocation of these skills rather than waiting for closed-source implementations, it effectively closes the performance gap long dominated by proprietary toolchains. The framework natively supports mainstream platforms including x86 and NVIDIA Orin SoCs, and represents the first end-to-end VLA deployment on the domestic Horizon Journey platform, enhancing cross-platform portability. In most cases, it yields clearly better inference performance than conventional vendor-specific toolchains, achieving up to 1.49 times speedup over TensorRT-Edge-LLM on the NVIDIA Orin platform. Experimental results show that EdgeFM delivers favorable end-to-end inference performance, providing an open-source, production-grade solution for diverse edge industrial scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。