用强化学习直接控推力矢量，让四旋翼飞得更稳更快。
Reinforcement Learning Position Control of a Quadrotor Using Soft Actor-Critic (SAC)
- 不直接调电机转速，而是通过推力矢量和姿态角控制飞行
- 训练速度更快，仿真中路径跟踪更平滑准确
- 适合想用强化学习提升无人机控制性能的研究者
本文提出一种基于强化学习（RL）的四旋翼控制架构。与以往直接控制四个螺旋桨转速不同，本方法通过控制四旋翼的推力矢量实现飞行控制。强化学习智能体计算出沿机体z轴的总推力占比，以及期望的滚转角（ϕ）和俯仰角（θ）。随后，将计算出的控制信号连同当前偏航角（ψ）发送给姿态PID控制器，由其将信号映射为电机转速。采用Soft Actor-Critic（SAC）算法进行训练，该算法为无模型、离策略的随机强化学习方法。训练结果表明，所提推力矢量控制器相比传统转速控制器具有更短的训练时间；仿真结果也显示，该控制器在路径跟踪上表现更平稳、更精确。
原文摘要 · Abstract (English)
This paper proposes a new Reinforcement Learning (RL) based control architecture for quadrotors. With the literature focusing on controlling the four rotors' RPMs directly, this paper aims to control the quadrotor's thrust vector. The RL agent computes the percentage of overall thrust along the quadrotor's z-axis along with the desired Roll ($ϕ$) and Pitch ($θ$) angles. The agent then sends the calculated control signals along with the current quadrotor's Yaw angle ($ψ$) to an attitude PID controller. The PID controller then maps the control signals to motor RPMs. The Soft Actor-Critic algorithm, a model-free off-policy stochastic RL algorithm, was used to train the RL agents. Training results show the faster training time of the proposed thrust vector controller in comparison to the conventional RPM controllers. Simulation results show smoother and more accurate path-following for the proposed thrust vector controller.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。