针对机器人低延迟推理，提出按动作偏差选择性验证，显著降低通信开销。
Action Deviation-Aware Inference for Low-Latency Wireless Robots
- 根据动作偏差决定是否上传并验证草案，减少无效通信。
- 实验显示传输量和服务器操作降40%，端到端延迟降39.2%。
- 适合需低延迟、高可靠性的移动机器人与工业控制场景。
为支持从自动驾驶到工业机器人操作等低延迟人工智能应用，6G愿景中分布式机器学习将移动、边缘与云的计算资源通过超可靠低延迟通信（HRLLC）连接。在此场景下，推测解码可实现分布式部署模型的协同推理：轻量级设备端模型本地生成草稿，远程更强大目标模型在并行推测采样过程中验证并修正草稿，从而在不牺牲准确率的前提下降低延迟。然而，不同于自回归文本生成，用于具身智能应用的行为克隆策略无法对多个草稿并行验证，因为每个生成动作依赖于前一动作更新的观测。为此，我们提出动作偏差感知混合推理（ADAHI），基于动作偏差选择性地传输与验证草稿，该偏差与目标模型拒绝该动作的概率高度相关。仅在必要时调用服务器操作，有效降低通信与计算开销，同时保留推测采样的准确率增益。测试平台实验表明，ADAHI使传输量与服务器操作减少约40%，端到端延迟降低39.2%，任务成功率可达基线（对每个草稿向量均启用推测采样）的97.2%。
原文摘要 · Abstract (English)
To support latency-sensitive AI applications ranging from autonomous driving to industrial robot manipulation, 6G envisions distributed ML with computational resources in mobile, edge, and cloud connected over hyper-reliable low-latency communication (HRLLC). In this setting, speculative decoding can facilitate collaborative inference of models distributively deployed: a lightweight on-device model locally generates drafts while a more capable remote target model on a server verifies and corrects them in parallel with speculative sampling, thus resulting in lower latency without compromising accuracy. However, unlike autoregressive text generation, behavior cloning policies, typically used for embodied AI applications, cannot parallelize verification and correction for multiple drafts as each generated action depends on observation updated by a previous action. To this end, we propose Action Deviation-Aware Hybrid Inference (ADAHI), wherein drafts are selectively transmitted and verified based on action deviation, which has a strong correlation with action's rejection probability by the target model. By invoking server operation only when necessary, communication and computational overhead can be reduced while accuracy gain from speculative sampling is preserved. Experiments on our testbed show that ADAHI reduces transmission and server operations by approximately 40%, lowers end-to-end latency by 39.2%, and attains up to 97.2% of the task-success rate of baseline that invokes speculative sampling for every draft embedding vector.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。