构建四类虚构机器人仿真环境，用于研究风格化运动控制的强化学习方法。
ARC-RL: A Reinforcement Learning Playground Inspired by ARC Raiders
- 基于游戏角色设计四类非真实机器人形态，统一接口与奖励函数。
- 仅用18个参数调节不同形态，实现跨形态快速迁移训练。
- 适合研究动画风格运动控制、离线先验数据提升的算法工程师。
用于腿式行走的强化学习已发展为由多组件奖励函数和物理引擎基准构成的体系，其形态均源于真实商用硬件。而游戏中的非玩家角色则受风格化约束，常呈现无真实机器人对应形态的生物体。我们提出ARC-RL，一个包含四个MuJoCo连续控制环境的套件，其机器人形态灵感源自《ARC Raiders》的生物图鉴：18自由度高型六足女王、12自由度装甲六足堡垒、18自由度紧凑六足跳虫和12自由度四足跃者。四类机器人共享统一观测模板、动作规范、仿真频率，以及单一闭式多组件奖励函数，仅通过少量权重和参数实现形态差异。该奖励函数融合速度追踪锥形项、存活健康奖励、相位锁定步态合规奖惩对、动作正则化项、三项安全惩罚及姿态锚定项，全程不使用动作捕捉数据。我们还为每种形态提供手工设计的中枢模式发生器示范，既作为固定专家参考，也作为离线到在线训练的先验数据源。在此平台上，我们开展对照实证研究，比较标准在线算法（SAC、SPEQ、SOPE-EO）与引入先验数据的方法（SACfD、SPEQ-O2O、SOPE），并分析各范式在应对形态多样性与动画风格约束时的表现。代码已公开于https://github.com/CarloRomeo427/ARC_RL.git。
原文摘要 · Abstract (English)
Reinforcement learning for legged locomotion has matured into a stack of multi-component reward functions and physics-engine benchmarks whose morphologies are uniformly derived from real commercial hardware. Game NPCs, however, are bound by stylistic constraints absent from sim-to-real robotics and routinely take the form of creatures with no real-robot counterpart. We introduce ARC-RL, a suite of four MuJoCo continuous-control environments featuring robotic morphologies inspired by the bestiary of ARC Raiders: the 18-DoF tall hexapod Queen, the 12-DoF armoured hexapod Bastion, the 18-DoF compact hexapod Tick, and the 12-DoF quadruped Leaper. All four robots share a unified observation template, action convention, simulation cadence, and a single closed-form multi-component reward function whose only per-morphology variation lives in a small set of weights and parameters. The reward fuses a velocity-tracking tent, a healthy survive bonus, a phase-locked gait-compliance bonus/cost pair, action regularisers, three safety penalties, and a posture anchor; no motion-capture data enters the reward at any point. We additionally provide hand-crafted Central Pattern Generator demonstrators per morphology, which serve both as fixed expert references and as sources of prior data for offline-to-online training. On this playground, we conduct a controlled empirical study comparing standard online algorithms (SAC, SPEQ, SOPE-EO) and methods augmented with prior data (SACfD, SPEQ-O2O, SOPE), and characterise how each paradigm copes with the playground's morphological diversity and animation-style stylistic constraints. Source code is available at https://github.com/CarloRomeo427/ARC_RL.git.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。