arXiv:2604.17701cs.ITcs.AI2026-04被引 1

针对边缘端大模型推理中猜测解码的通信瓶颈,提出无线感知语义验证机制。

WISV: Wireless-Informed Semantic Verification for Distributed Speculative Decoding in Device-Edge LLM Inference

论文配图:WISV: Wireless-Informed Semantic Verification for Distributed Speculative Decoding in Device-Edge LLM Inference
图 1 · 摘自论文原文
  • 根据信道状态动态评估猜测词,突破严格逐词匹配限制
  • 实测接受序列长度提升60.8%,交互轮次减少37.3%,端到端延迟降31.4%
  • 适用于资源受限边缘设备部署,兼顾精度与实时性

分布式设备-边缘猜测解码虽能提升异构节点资源利用率,但传统逐词验证策略在波动无线环境下常因过度拒绝导致有效序列长度缩短、交互轮次增加。本文提出WISV(无线感知语义验证)框架,通过融合瞬时信道状态信息(CSI)与高维隐藏表示,设计面向信道感知的语义接受策略。在边缘侧目标大模型中引入轻量级决策头,动态评估猜测词。为平衡验证精度与通信开销,进一步设计全隐藏上传与错位优先选择性上传两种协议。基于1B起草模型与8B目标模型的大量仿真表明,相比基线方案,WISV在各测试场景下实现接受序列长度最高提升60.8%、交互轮次降低37.3%、端到端延迟减少31.4%,任务准确率损失小于1%。硬件测试平台(NVIDIA Jetson AGX Orin + A40服务器)验证了其在真实边缘环境中的有效性。

原文摘要 · Abstract (English)

While distributed device-edge speculative decoding enhances resource utilization across heterogeneous nodes, its performance is often bottlenecked by conventional token-level verification strategies. Such rigid alignment leads to excessive rejections, significantly diminishing the accepted sequence length and increasing interaction rounds under fluctuating wireless conditions. In this paper, we propose WISV (Wireless-Informed Semantic Verification), a novel distributed speculative decoding framework that goes beyond strict token-level matching via a channel-aware semantic acceptance policy. WISV integrates a lightweight decision head into the edge-side target LLM to dynamically evaluate speculative tokens by synthesizing high-dimensional hidden representations with instantaneous channel state information (CSI). To optimize the trade-off between verification fidelity and communication overhead, we further design two tailored communication protocols: full-hidden upload and mismatch-first selective-hidden upload. Extensive simulations using a 1B drafter and an 8B target model demonstrate that WISV achieves up to a 60.8% increase in accepted length, a 37.3% reduction in interaction rounds, and a 31.4% improvement in end-to-end latency compared to vanilla speculative decoding across tested settings, while maintaining a negligible task accuracy drop (<1%). Finally, we validate WISV on a hardware testbed comprising an NVIDIA Jetson AGX Orin and an A40-equipped server, confirming its real-world efficacy in accelerating edge-deployed LLM inference.

边缘计算大模型推理猜测解码无线优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。