arXiv:2607.26865stat.MLcs.AI2026-07

边端大模型推理新框架,省算力还敢放心用。

Think Short, Defer Smart, Act, and Repeat: Calibrated Reasoning and Uncertainty-Aware Deferral for Edge LLM Agents

论文配图:Think Short, Defer Smart, Act, and Repeat: Calibrated Reasoning and Uncertainty-Aware Deferral for Edge LLM Agents
图 1 · 摘自论文原文
  • 用轻量收敛探测器提前终止推理,省下大量计算
  • 根据困惑度智能判断何时把任务转给云端,降低调用次数43%-65%
  • 兼顾性能与安全,适合部署在资源受限的边缘设备上

遵循ReAct范式的大型语言模型代理是完成复杂多步任务的有力工具,涵盖多跳问答、代码生成及物理系统控制等。然而在边缘部署时,必须在有限推理预算内保持可靠性,并仅在本地不确定性过高时才将任务回传云端。本文提出Think Short, Defer Smart(TSDS)框架,融合轻量级收敛探测器与基于困惑度的智能回传机制,通过多目标学习-测试(LTT)流程联合校准,实现对每轮任务预期奖励与云端调用率的有限样本保证。在涵盖算术推理(GSM8K)、多跳问答(HotpotQA)、代码生成(MBPP)和多步具身规划(household robot)的四个基准上评估,TSDS相较仅校准思考或仅支持回传的基线,在HotpotQA、MBPP及家庭机器人任务中减少43%~65%的每轮推理计算量,同时维持经认证的奖励与云端调用率表现。

原文摘要 · Abstract (English)

LLM agents following the ReAct paradigm are promising enablers of complex multi-step tasks, including multi-hop question answering, code generation, and control of physical AI systems. Yet, when deployed at the edge, they must tightly manage their reasoning budget while remaining reliable and deferring to a cloud-side model only when local uncertainty is too high to act safely. We propose Think Short, Defer Smart (TSDS), a framework that synergistically integrates a lightweight convergence probe, which halts on-device reasoning once the intended action has stabilized, with a perplexity-based deferral rule that escalates uncertain actions to a cloud-side model. Both mechanisms are jointly calibrated on end-to-end episode trajectories via a multi-objective Learn-Then-Test (LTT) procedure, providing simultaneous finite-sample guarantees on expected episode reward and cloud-call rate. We evaluate TSDS on four ReAct benchmarks spanning arithmetic reasoning (GSM8K), multi-hop question answering (HotpotQA), code generation (MBPP), and multi-step embodied planning (household robot), and compare against thought-calibration-only and calibrated-deferral-only standalone baselines. TSDS reduces per-episode thinking compute by 43%-65% over deferral-only baselines across HotpotQA, MBPP, and the household robot task, while maintaining certified reward and cloud-call rate guarantees.

边缘计算大模型推理优化智能回传

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。