用可插拔的动态滤波器，让旧信息变实时，提升多智能体系统通信延迟下的表现。
Decoupled Delay Compensation: Enhancing Pre-trained MARL Policies via Learned Dynamics Filtering
- 引入门控状态转移与递归卡尔曼滤波，从异步观测中估计当前状态。
- 在多种连续控制任务中，延迟和丢包场景下性能显著提升。
- 无需修改训练流程，可直接插入已有策略，适合真实部署场景。
现实中的多智能体强化学习系统常面临观测过时、通信随机延迟和间歇性丢包问题。在理想同步条件下训练的策略在这些场景下性能显著下降，因其依赖过时反馈。本文提出一种模块化的执行阶段状态估计层，将延迟传递的观测替换为当前信念状态估计。该框架结合了学习得到的门控转移模型与递归卡尔曼滤波层，从异步测量中推断瞬时状态。其主要优势在于模块化：该估计器可作为即插即用组件集成至预训练策略中，无需修改原始MARL训练算法、网络结构或奖励设计。在多个多智能体及连续控制基准测试中，所提方法均显著增强对通信延迟和消息丢失的鲁棒性。在依赖协调与动态不稳定的任务中，性能提升最为明显，说明时间一致性对控制至关重要。
原文摘要 · Abstract (English)
Real-world multi-agent reinforcement learning (MARL) systems must often operate under stale observations, stochastic communication delays, and intermittent packet loss. Policies trained under idealized synchronous conditions frequently exhibit significant performance degradation in these regimes because they act on outdated feedback. We propose a modular execution-stage state-estimation layer that replaces delayed communicated observations with current belief-state estimates. The framework integrates a learned Gated transition model with a recursive Kalman filtering layer to estimate instantaneous states from asynchronous measurements. A primary advantage of this approach is its modularity, The estimator serves as a plug-in for pre-trained policies, requiring no modifications to the original MARL training algorithm, architecture, or reward structure. Evaluation across diverse multi-agent and continuous-control benchmarks demonstrates that the proposed layer consistently enhances robustness to communication latency and message loss. The most significant performance gains are observed in coordination-intensive and dynamically unstable tasks where temporal consistency is critical for control.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。