arXiv:2605.16205cs.AIcs.CL2026-05被引 1

在对抗性环境里,优化任务分解比深度推理更有效。

Context, Reasoning, and Hierarchy: A Cost-Performance Study of Compound LLM Agent Design in an Adversarial POMDP

论文配图:Context, Reasoning, and Hierarchy: A Cost-Performance Study of Compound LLM Agent Design in an Adversarial POMDP
图 1 · 摘自论文原文
  • 用程序化状态抽象替代原始观测,提升效率
  • 分层推理反而导致性能下降,最高差3.4倍
  • 任务拆分清晰+上下文设计,性价比更高

在对抗性、部分可观测的序列环境中部署复合大模型智能体,需权衡三个设计维度:感知内容、推理方式与任务分解结构。然而,实践中缺乏对性能提升与推理成本之间关系的指导。本文在CybORG CAGE-2(一个建模为部分可观测马尔可夫决策过程POMDP的网络防御环境)中开展受控研究,评估五类模型、六种模型及十二种配置(共3,475轮)。奖励非正数,所有配置均处于故障缓解模式。研究对比了上下文表示(原始观测 vs. 压缩历史的状态追踪层)、推理机制(自问自答、自检、自我改进工具,含可选思维链提示)以及层级分解(单体ReAct vs. 专业子智能体委派)。结果表明:(1) 程序化状态抽象的每令牌回报(RPTS)最高,使平均回报提升76%;(2) 将推理工具分布于层级结构中会引发性能恶化,最差时平均回报降低3.4倍,同时消耗1.8–2.7倍更多令牌,称此为“推理级联”;(3) 无推理的层级分解在多数模型上表现最佳,且上下文工程普遍比推理更高效。结论建议:针对结构化对抗性POMDP,应优先投入程序化基础设施与清晰任务分解,而非深化个体推理,因二者组合可能产生干扰。

原文摘要 · Abstract (English)

Deploying compound LLM agents in adversarial, partially observable sequential environments requires navigating several design dimensions: (1) what the agent sees, (2) how it reasons, and (3) how tasks are decomposed across components. Yet practitioners lack guidance on which design choices improve performance versus merely increase inference costs. We present a controlled study of compound LLM agent design in CybORG CAGE-2, a cyber defense environment modeled as a Partially Observable Markov Decision Process (POMDP). Reward is non-positive, so all configurations operate in a failure-mitigation mode. Our evaluation spans five model families, six models, and twelve configurations (3,475 episodes) with token-level cost accounting. We vary context representation (raw observations vs. a deterministic state-tracking layer with compressed history), deliberation (self-questioning, self-critique, and self-improvement tools, with optional chain-of-thought prompting), and hierarchical decomposition (monolithic ReAct vs. delegation to specialized sub-agents). We find that: (1) Programmatic state abstraction delivers the largest returns per token spent (RPTS), improving mean return by up to 76% over raw observations. (2) Distributing deliberation tools across a hierarchy degrades performance relative to hierarchy alone for all five model families, reaching up to 3.4$\times$ worse mean return while using 1.8-2.7$\times$ more tokens. We call this destructive pattern a deliberation cascade. (3) Hierarchical decomposition without deliberation achieves the best absolute performance for most models, and context engineering is generally more cost-effective than deliberation. These findings suggest a design principle for structured adversarial POMDPs: invest in programmatic infrastructure and clean task decomposition rather than deeper per-agent reasoning, as these strategies can interfere when combined.

大模型代理强化学习系统设计推理效率

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。