用自适应束搜索提升无人机机械臂在扰动下的控制精度
Meta-Adaptive Beam Search Planning for Transformer-Based Reinforcement Learning Control of UAVs with Overhead Manipulators under Flight Disturbances
- 基于Transformer的强化学习框架，通过束搜索预演动作序列
- 跟踪误差降至3%，奖励提升10.2%，综合指标改善29.6%
- 适合受扰动影响的空中机械臂控制，稳定性更强
搭载顶置机械臂的无人机可执行巡检、维护及接触式操作，但飞行器姿态变化会显著影响末端执行器轨迹。为应对这一耦合问题，本文提出一种基于Transformer的双深度Q网络（DDQN）强化学习框架，核心是采用元自适应束搜索规划器，在短时程内对候选控制序列进行模拟推演，利用学习到的评判器作为前向估计器，实现软件在环（SITL）控制。该方法通过价值估计预测未来状态，结合DDQN保持训练稳定。在3-自由度空中机械臂上测试，相较基准DDQN，本方法奖励提升10.2%，平均跟踪误差从约6%降至3%，综合指标改善29.6%。在外部扰动导致基座漂移时，仍能维持5厘米以内的跟踪误差，优于固定束宽与纯Transformer变体。
原文摘要 · Abstract (English)
Drones equipped with overhead manipulators offer unique capabilities for inspection, maintenance, and contact-based interaction. However, the motion of the drone and its manipulator is tightly linked, and even small attitude changes caused by wind or control imperfections shift the end-effector away from its intended path. This coupling makes reliable tracking difficult and also limits the direct use of learning-based arm controllers that were originally designed for fixed-base robots. These effects appear consistently in our tests whenever the UAV body experiences drift or rapid attitude corrections. To address this behavior, we develop a reinforcement-learning (RL) framework with a transformer-based double deep Q learning (DDQN), with the core idea of using an adaptive beam-search planner that applies a short-horizon beam search over candidate control sequences using the learned critic as the forward estimator. This allows the controller to anticipate the end-effector's motion through simulated rollouts rather than executing those actions directly on the actual model, realizing a software-in-the-loop (SITL) approach. The lookahead relies on value estimates from a Transformer critic that processes short sequences of states, while a DDQN backbone provides the one-step targets needed to keep the learning process stable. Evaluated on a 3-DoF aerial manipulator under identical training conditions, the proposed meta-adaptive planner shows the strongest overall performance with a 10.2% reward increase, a substantial reduction in mean tracking error (from about 6% to 3%), and a 29.6% improvement in the combined reward-error metric relative to the DDQN baseline. Our method exhibits elevated stability in tracking target tip trajectory (by maintaining 5 cm tracking error) when the drone base exhibits drifts due to external disturbances, as opposed to the fixed-beam and Transformer-only variants.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。