针对边缘设备推理，提出一种成本感知的风险可控路由方法。
CR^2: Cost-Aware Risk-Controlled Routing for Wireless Device-Edge LLM Inference
- 分两阶段路由：设备端判断是否本地执行，边缘端处理延迟请求。
- 在相同准确率下，部署成本降低最多16.9%。
- 支持显式控制错误接受风险，适合资源受限的移动边缘场景。
随着大语言模型从中心云向移动边缘环境迁移，高效服务需在有限的设备-边缘资源下平衡延迟、能耗与精度。查询级路由在轻量级设备模型与更强边缘模型间灵活切换，可有效权衡性能。然而，现有路由器多针对中心云设计，优化目标为令牌级开销，难以捕捉无线边缘部署中的动态延迟与能耗开销。本文将移动边缘LLM路由建模为受部署约束的成本感知决策问题，提出CR^2——一种两阶段设备-边缘路由框架。该框架将轻量级设备端的边际判断门与边缘端的效用选择器解耦，用于延迟查询处理。边际门基于冻结的查询嵌入与用户指定的成本权重，预测本地执行相对于最佳边缘方案在目标运行点下的效用最优性。我们进一步引入共形风险控制（CRC）校准流程，将每个运行点映射为接受阈值，实现对全信息效用参考下的边际误接受风险的显式控制。实验表明，仅使用设备端信号，CR^2即可接近全信息参考路由器的表现。相较于强基准，CR^2持续提升可部署的准确率-成本帕累托前沿，并在匹配准确率下将归一化部署成本降低最高达16.9%。
原文摘要 · Abstract (English)
As large language models (LLMs) move from centralized clouds to mobile edge environments, efficient serving must balance latency, energy consumption, and accuracy under constrained device-edge resources. Query-level routing between lightweight on-device models and stronger edge models provides a flexible mechanism to navigate this trade-off. However, existing routers are designed for centralized cloud settings and optimize token-level costs, failing to capture the dynamic latency and energy overheads in wireless edge deployments. In this paper, we formulate mobile edge LLM routing as a deployment-constrained, cost-aware decision problem, and propose CR^2, a two-stage device-edge routing framework. CR^2 decouples a lightweight on-device margin gate from an edge-side utility selector for deferred queries. The margin gate operates on frozen query embeddings and a user-specified cost weight to predict whether local execution is utility-optimal relative to the best edge alternative under the target operating point. We further introduce a conformal risk control (CRC) calibration procedure that maps each operating point to an acceptance threshold, enabling explicit control of the marginal false-acceptance risk under the full-information utility reference. Experiments on the routing task show that CR^2 closely matches a full-information reference router using only device-side signals before deferral. Compared with strong query-level baselines, CR^2 consistently improves the deployable accuracy-cost Pareto frontier and reduces normalized deployment cost by up to 16.9% at matched accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。