arXiv:2605.19592cs.ROcs.AI2026-05

提出隐式动作分块方法,让强化学习控制更平滑稳定。

Implicit Action Chunking for Smooth Continuous Control

论文配图:Implicit Action Chunking for Smooth Continuous Control
图 1 · 摘自论文原文
  • 用双窗口机制隐式实现动作分块,不扩大动作空间。
  • 在自动驾驶任务中实现100%成功率,抖动减少且行为更安全。
  • 适合需要平稳控制的物理系统部署,如机器人和能源管理。

强化学习常产生高频振荡的控制信号,影响物理部署的安全与稳定。显式动作分块虽能缓解此问题,但会随时间窗增长增加策略输出维度,导致优化困难且不兼容标准步进交互。本文提出双窗口平滑(DWS)框架,一种隐式动作分块方法,无需扩展动作空间即可实现时间一致性。DWS采用双窗口设计:执行窗口通过确定性调制确保物理平滑;价值窗口通过跨时域对齐时序差分目标,纠正开环执行带来的评价偏差。此外,引入轻量级演员端一阶动作差分正则项,促进全局连续性。该设计有效弥合了时间抽象与反应式步进控制之间的鸿沟。在DeepMind Control Suite及工业能源管理任务上的实验表明,DWS优于现有最先进方法。在复杂视觉驱动的自动驾驶任务中,DWS实现更平滑控制、更少抖动、更高安全性,并达到100%成功率。

原文摘要 · Abstract (English)

Reinforcement learning often produces high-frequency oscillatory control signals that undermine the safety and stability required for physical deployment. Explicit action chunking addresses this by predicting fixed-horizon trajectories but scales the policy output dimension proportionally with the horizon length, leading to optimization difficulties and incompatibility with standard step-wise interaction. To overcome these challenges, this paper proposes Dual-Window Smoothing (DWS), an implicit action chunking framework for smooth continuous control. Unlike explicit methods, DWS enforces temporal coherence without expanding the action space. It uses a dual-window design: an execution window that ensures physical smoothness through deterministic modulation, and a value window that aligns temporal-difference targets over the horizon to correct critic bias caused by open-loop execution. DWS also includes a lightweight actor-side temporal regularizer based on first-order action differences to promote global continuity. This design effectively bridges the gap between temporal abstraction and reactive step-wise control. Experiments on benchmarks including the DeepMind Control Suite and industrial energy management tasks show that DWS outperforms state-of-the-art (SOTA) baselines. In complex vision-based autonomous driving tasks, DWS achieves smoother control, safer behavior with reduced jitter, and attains a 100% success rate.

强化学习连续控制动作分块自动驾驶

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。