用隐空间动作重参数化,让大模型智能体更高效决策
Latent Action Reparameterization for Efficient Agent Inference

- 将动作空间映射到紧凑隐变量,每个动作代表多步语义行为
- 减少有效决策时长70%以上,推理时间缩短且成功率不降
- 适合追求推理效率的智能体系统开发人员
大型语言模型(LLM)智能体通常依赖于长序列的低级文本动作,导致有效决策时长远、推理成本高。现有工作多聚焦于系统优化或提示工程提升效率,但我们认为动作空间表示本身是关键瓶颈。本文提出隐动作重参数化(LAR),学习一个紧凑的隐动作空间,其中每个隐动作对应多步语义行为。通过将动作重参数化为隐单元,LAR实现更短的有效决策时长,同时保持原始动作空间的表达能力。与手工设计的宏指令或分层控制器不同,隐动作从智能体轨迹中学习,并直接嵌入模型,使规划与执行均在抽象动作表示上进行。在多个基于LLM的智能体基准测试中,LAR显著降低有效动作时长,在固定计算预算下大幅提升推理效率。结果表明,该方法在减少动作令牌数和实际推理时间方面表现优异,且任务成功率维持或提升。这说明动作表示学习是扩展高效LLM智能体推理的关键因素,与模型架构和硬件进步相辅相成。
原文摘要 · Abstract (English)
Large language model (LLM) agents often rely on long sequences of low-level textual actions, resulting in large effective decision horizons and high inference cost. While prior work has focused on improving inference efficiency through system-level optimizations or prompt engineering, we argue that a key bottleneck lies in the representation of the action space itself. We propose Latent Action Reparameterization (LAR), a framework that learns a compact latent action space in which each latent action corresponds to a multi-step semantic behavior. By reparameterizing agent actions into latent units, LAR enables decision making over a shorter effective horizon while preserving the expressiveness of the original action space. Unlike hand-crafted macros or hierarchical controllers, latent actions are learned from agent trajectories and integrated directly into the model, allowing both planning and execution to operate over abstract action representations. Across a range of LLM-based agent benchmarks, LAR significantly reduces the effective action horizon and improves inference efficiency under fixed compute budgets. As a consequence, our approach achieves substantial reductions in action tokens and corresponding wall-clock inference time, while maintaining or improving task success rates. These results suggest that action representation learning is a critical and underexplored factor in scaling efficient LLM agent inference, complementary to advances in model architecture and hardware.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。