MARLIN用多智能体强化学习高效发现因果图结构。
MARLIN: Multi-Agent Reinforcement Learning for Incremental DAG Discovery
- 设计双智能体架构，分别捕捉状态相关与无关的因果关系。
- 在合成与真实数据集上，效率和准确率均优于现有方法。
- 适合需要实时更新因果关系的动态系统分析场景。
从观测数据中揭示因果结构对理解复杂系统和做出明智决策至关重要。尽管强化学习（RL）在以有向无环图（DAG）形式识别这些结构方面展现出潜力，但现有方法通常效率低下，难以用于在线应用。本文提出MARLIN，一种基于多智能体强化学习的增量DAG学习高效方法。MARLIN采用将连续实值空间映射到DAG空间的生成策略作为批内机制，引入两个强化学习智能体——状态特定与状态不变——以发现因果关系，并将其整合进增量学习框架。此外，该框架利用因子化动作空间提升并行化效率。在合成与真实数据集上的大量实验表明，MARLIN在效率和有效性方面均优于当前最优方法。
原文摘要 · Abstract (English)
Uncovering causal structures from observational data is crucial for understanding complex systems and making informed decisions. While reinforcement learning (RL) has shown promise in identifying these structures in the form of a directed acyclic graph (DAG), existing methods often lack efficiency, making them unsuitable for online applications. In this paper, we propose MARLIN, an efficient multi agent RL based approach for incremental DAG learning. MARLIN uses a DAG generation policy that maps a continuous real valued space to the DAG space as an intra batch strategy, then incorporates two RL agents state specific and state invariant to uncover causal relationships and integrates these agents into an incremental learning framework. Furthermore, the framework leverages a factored action space to enhance parallelization efficiency. Extensive experiments on synthetic and real datasets demonstrate that MARLIN outperforms state of the art methods in terms of both efficiency and effectiveness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。