通过约束流形控制实现安全且可泛化的多智能体强化学习
Safe and Generalizable Hierarchical Multi-Agent RL via Constraint Manifold Control
- 低层用约束流形强制安全，高层用策略学习实现协调
- 实测近满分安全率，且在不同智能体数量下表现稳定
- 适合需要高安全性的多智能体系统，如自动驾驶编队
多智能体系统广泛应用于需严格安全约束的场景。现有方法存在根本性权衡：基于学习的方法性能强但缺乏理论安全保障，控制理论方法虽能保证安全却常导致过度保守与低效。本文提出一种分层多智能体强化学习框架，通过约束流形在低层施加硬性安全约束，同时在高层通过策略学习实现有效协同。该方法在多智能体设置中提供理论安全保证，并实现平稳的学习动态，从而支持稳定高效的训练。实验表明，该方法在保持接近100%安全率的同时，达到具有竞争力的性能，并能有效泛化至不同数量的智能体和障碍物。
原文摘要 · Abstract (English)
Multi-agent systems are widely used in safety-critical applications that require coordinated behavior under strict safety constraints. Existing approaches face a fundamental trade-off: learning-based methods achieve strong empirical performance but lack theoretical safety guarantees, while control-theoretic methods enforce safety but often lead to overly conservative and inefficient behaviors. We propose a hierarchical multi-agent reinforcement learning framework that enforces hard safety constraints under mild assumptions at low level via a constraint manifold, while enabling effective coordination through high-level policy learning. Our approach provides theoretical safety guarantees in the multi-agent setting and yields stationary learning dynamics, thereby enabling stable and efficient training. Empirically, our method achieves competitive performance while maintaining nearly perfect safety rates, and generalizes effectively to varying numbers of agents and obstacles.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。