arXiv:2603.23722cs.MAcs.LG2026-03

让多智能体在复杂任务中自动调节计算频率,省电又高效。

Dual-Gated Epistemic Time-Dilation: Autonomous Compute Modulation in Asynchronous MARL

  • 根据策略不确定性与状态价值差异动态调整推理频率
  • 在谷歌足球数据集上减少73.6%计算开销,性能提升超60%
  • 适合资源受限的边缘设备部署,支持异步学习机制

尽管多智能体强化学习(MARL)在复杂连续领域取得突破,但其标准部署仍采用同步模式:所有智能体每帧必须执行深度神经网络推理,无论是否必要。这种高密度计算严重制约了边缘设备的物理部署,因热耗和能耗预算极为有限。本文提出基于双门控认知触发的认知时滞MAPPO(ETD-MAPPO),不再依赖固定的帧跳过机制,而是通过分析策略的随机不确定性(以策略熵衡量)和认知不确定性(基于孪生评判器的状态值差异)自主调节执行频率。为此,将环境建模为半马尔可夫决策过程(SMDP),并构建适配的异步梯度掩码评判器以保证信用分配准确。实验表明,在谷歌研究足球(GRF)115维状态空间、LBF与MPE基准测试中,该方法相比现有时间模型实现超过60%的性能跃升,并有效防止策略过早崩溃。令人惊讶的是,这一无约束方法催生了时间角色分工,仅在非持球阶段就降低了73.6%的计算负载,同时维持中央任务主导能力。

原文摘要 · Abstract (English)

While Multi-Agent Reinforcement Learning (MARL) algorithms achieve unprecedented successes across complex continuous domains, their standard deployment strictly adheres to a synchronous operational paradigm. Under this paradigm, agents are universally forced to execute deep neural network inferences at every micro-frame, regardless of immediate necessity. This dense throughput acts as a fundamental barrier to physical deployment on edge-devices where thermal and metabolic budgets are highly constrained. We propose Epistemic Time-Dilation MAPPO (ETD-MAPPO), augmented with a Dual-Gated Epistemic Trigger. Instead of depending on rigid frame-skipping (macro-actions), agents autonomously modulate their execution frequency by interpreting aleatoric uncertainty (via Shannon entropy of their policy) and epistemic uncertainty (via state-value divergence in a Twin-Critic architecture). To format this, we structure the environment as a Semi-Markov Decision Process (SMDP) and build the SMDP-Aligned Asynchronous Gradient Masking Critic to ensure proper credit assignment. Empirical findings demonstrate massive improvements (> 60% relative baseline acquisition leaps) over current temporal models. By assessing LBF, MPE, and the 115-dimensional state space of Google Research Football (GRF), ETD correctly prevented premature policy collapse. Remarkably, this unconstrained approach leads to emergent Temporal Role Specialization, reducing computational overhead by a statistically dominant 73.6% entirely during off-ball execution without deteriorating centralized task dominance.

多智能体强化学习边缘计算自适应推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。