根据输入复杂度动态分配视觉语言任务，兼顾速度与精度。
INAR-VL: Input-Aware Routing for Edge-Cloud Vision-Language Inference
- 基于图像和文本复杂度轻量级判断，智能分配任务到边缘或云端
- 边缘执行36%请求，延迟降24%，能耗减26%，准确率保留97%
- 适合对实时性与能效敏感的多模态应用，如移动端视觉问答
边缘部署视觉语言模型（VLMs）面临延迟与精度的权衡：云端推理精度高但通信延迟大、能耗高；边缘本地推理虽快但受限于算力，准确率较低。这一权衡因图像质量与推理复杂度异质性而加剧，静态部署难以优化。本文提出INAR-VL，一种轻量级的边云协同多模态推理路由系统。该系统在边缘与云端并行部署互补的VLM，并利用轻量级图像与文本复杂度信号，引导任务路由与模型选择——简单任务本地处理，复杂任务按需卸载。在视觉问答任务上的评估表明，INAR-VL有36%的请求在边缘执行，平均延迟降低24%，能耗减少26%，同时保持97%的云端精度水平。
原文摘要 · Abstract (English)
Edge deployment of Vision-Language Models (VLMs) faces a tradeoff between latency and accuracy: cloud execution provides high-quality predictions but incurs communication delay and energy cost, while edge-only execution is faster but less accurate due to limited model capacity. This trade-off is further complicated by heterogeneity in image quality and reasoning complexity, making static placement suboptimal. We present INAR-VL, a lightweight edge-cloud routing system for multimodal inference in a two-tier deployment. INAR-VL maintains complementary VLMs across edge and cloud and uses lightweight image and text complexity signals to guide routing and model selection, executing simple queries locally while offloading complex ones when beneficial. Evaluation on visual question answering shows that INAR-VL executes 36% of requests on the edge, reduces latency by 24%, lowers energy by 26%, and preserves 97% of cloud-level accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。