提出动态选择动作块的新算法，兼顾实时反应与决策一致性。
Temporal Action Selection for Action Chunking
- 缓存多时间步动作预测，用轻量网络动态选最优动作
- 实验显示成功率显著提升，且训练效率与性能上限均提高
- 适合需要快速响应的机器人控制场景
动作块化是学习演示（LfD）中的常用方法，通过建模多步动作块而非单步动作，显著增强对人类专家策略的建模能力。然而，由于动作块化仅在完整动作序列执行后才做一次决策，导致决策频率降低，限制了对实时观测的利用，在动态或噪声环境中影响反应能力。现有方法主要在反应性与决策一致性之间权衡，难以兼顾。为此，我们提出一种新算法——时序动作选择（Temporal Action Selection, TAS），通过缓存多个时间步的预测动作块，并利用轻量选择网络动态选取最优动作，实现反应性与决策一致性的平衡优化。在多种任务和基线策略架构上的实验表明，TAS 显著提升成功率。进一步将 TAS 作为基线策略结合残差强化学习（RL），可同时提升训练效率与性能上限。仿真与物理机器人实验均验证了该方法的有效性。
原文摘要 · Abstract (English)
Action chunking is a widely adopted approach in Learning from Demonstration (LfD). By modeling multi-step action chunks rather than single-step actions, action chunking significantly enhances modeling capabilities for human expert policies. However, because action chunking makes a single decision only after a complete action block has been executed, the resulting reduction in decision frequency restricts the utilization of real-time observations, impairing reactivity in dynamic or noisy environments. Existing efforts to address this issue have primarily resorted to trading off reactivity against decision consistency, without achieving both. To address this limitation, we propose a novel algorithm, Temporal Action Selection (TAS), which caches predicted action chunks from multiple timesteps and dynamically selects the optimal action through a lightweight selector network. TAS achieves balanced optimization across both reactivity and decision consistency. Experiments across multiple tasks with diverse base policy architectures show that TAS significantly improves success rates. Furthermore, integrating TAS as a base policy with residual reinforcement learning (RL) improves both training efficiency and the performance ceiling. Experiments in both simulation and physical robots confirm the method's efficacy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。