arXiv:2511.08086cs.LGcs.AI2025-11AAAI被引 2

实证发现机器人任务动态具有状态依赖的局部稀疏性,挑战了普遍假设的全局稀疏性。

Dynamic Sparsity: Challenging Common Sparsity Assumptions for Learning World Models in Robotic Reinforcement Learning Benchmarks

  • 通过分析MuJoCo Playground基准的真实动力学,发现状态变量间依赖关系是局部且随状态变化的。
  • 动态稀疏性主要出现在接触等瞬时事件中,影响特定状态维度,而非全局稀疏。
  • 适合关注真实世界动力学建模与强化学习中先验设计的研究者。

学习动态模型(即世界模型)可提升强化学习的样本效率。近期研究认为这些模型的因果图具有稀疏连接,未来状态变量仅依赖当前状态的少数部分,因此引入稀疏性先验可能有益。类似地,时间稀疏性(即局部动力学稀疏且突变)也被视为有用的归纳偏置。本文通过分析MuJoCo Playground基准套件中一系列机器人强化学习环境的真实动力学,检验这些稀疏性假设是否在典型任务中成立。我们研究:(i) 环境动力学的因果图是否稀疏;(ii) 这种稀疏性是否依赖于状态;(iii) 局部系统动态是否稀疏变化。结果表明,全局稀疏性罕见,但任务呈现局部、状态依赖的稀疏性,其结构具有明显特征:在特定时刻(如接触事件)集中出现,并影响特定状态维度。这一发现挑战了动态学习中常见的稀疏性先验假设,强调需建立基于真实动态结构的状态依赖先验。

原文摘要 · Abstract (English)

The use of learned dynamics models, also known as world models, can improve the sample efficiency of reinforcement learning. Recent work suggests that the underlying causal graphs of such dynamics models are sparsely connected, with each of the future state variables depending only on a small subset of the current state variables, and that learning may therefore benefit from sparsity priors. Similarly, temporal sparsity, i.e. sparsely and abruptly changing local dynamics, has also been proposed as a useful inductive bias. In this work, we critically examine these assumptions by analyzing ground-truth dynamics from a set of robotic reinforcement learning environments in the MuJoCo Playground benchmark suite, aiming to determine whether the proposed notions of state and temporal sparsity actually tend to hold in typical reinforcement learning tasks. We study (i) whether the causal graphs of environment dynamics are sparse, (ii) whether such sparsity is state-dependent, and (iii) whether local system dynamics change sparsely. Our results indicate that global sparsity is rare, but instead the tasks show local, state-dependent sparsity in their dynamics and this sparsity exhibits distinct structures, appearing in temporally localized clusters (e.g., during contact events) and affecting specific subsets of state dimensions. These findings challenge common sparsity prior assumptions in dynamics learning, emphasizing the need for grounded inductive biases that reflect the state-dependent sparsity structure of real-world dynamics.

世界模型强化学习稀疏性机器人

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。