用更丰富的状态信息让强化学习超越传统控制，实现无人机球平衡
Rich State Observations Empower Reinforcement Learning to Surpass PID: A Drone Ball Balancing Study
- 分层控制：高层用强化学习做决策，底层用传统控制
- 强化学习在模拟中优于调优后的PID控制器
- 优势来自更全面的状态观测，而非算法本身
本文研究无人机通过缆绳控制可移动横梁上的小球保持平衡的任务。提出一种分层控制框架，将高层平衡策略与低层无人机控制解耦，训练强化学习（RL）策略负责高层决策。仿真结果显示，在相同分层结构下，该RL策略性能优于精心调校的PID控制器。通过系统性对比分析，发现其优势并非源于参数调优或非线性映射能力，而是能够有效利用更丰富的状态观测信息。结果表明，全面的状态表征对学习型系统至关重要，增强感知可能显著提升控制器性能。
原文摘要 · Abstract (English)
This paper addresses a drone ball-balancing task, in which a drone stabilizes a ball atop a movable beam through cable-based interaction. We propose a hierarchical control framework that decouples high-level balancing policy from low-level drone control, and train a reinforcement learning (RL) policy to handle the high-level decision-making. Simulation results show that the RL policy achieves superior performance compared to carefully tuned PID controllers within the same hierarchical structure. Through systematic comparative analysis, we demonstrate that RL's advantage stems not from improved parameter tuning or inherent nonlinear mapping capabilities, but from its ability to effectively utilize richer state observations. These findings underscore the critical role of comprehensive state representation in learning-based systems and suggest that enhanced sensing could be instrumental in improving controller performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。