用共享未来状态解决连续规划中搜索爆炸问题
Graph Sparse Sampling: Breaking the Curse of the Horizon in Continuous MDP Planning

- 构建无分支图结构,复用采样轨迹覆盖多个决策路径
- 在长时序规划中性能远超传统树形方法,接近最优解
- 适合需要长期推理的机器人控制场景
连续域中的不确定性规划对自主系统至关重要,但计算成本高昂。基于树的搜索方法如蒙特卡洛树搜索(MCTS)在最坏情况下,其分支数随前瞻深度呈指数增长。在连续状态或动作空间中,规划器需决定在无限分支层级中何处搜索。本文提出图稀疏采样(GSS),一种在线规划算法,通过共享采样未来状态,避免为每个候选动作单独采样后继节点。该无分支图结构支持大规模并行计算,同时利用启发式聚焦计算资源。我们证明了在全秩或低秩生成模拟器下,GSS具有有限样本性能保证,适用于离散或采样连续动作空间。在适当重叠、平滑性及动作覆盖条件下,其性能界对规划时序呈多项式依赖,证明共享未来可避免树形稀疏采样中的指数依赖。实验显示,在长时序连续控制任务中,GSS显著优于树基规划器,或达到近优性能,验证了无分支图规划作为在线控制的互补设计原则。
原文摘要 · Abstract (English)
Planning under uncertainty in continuous domains is essential for autonomous systems, yet computationally demanding. Tree-based search methods such as Monte Carlo Tree Search (MCTS) remain popular, but their branching structure can require sampling budgets that grow exponentially with lookahead depth in the worst case. From a tree perspective, continuous state or action spaces become especially challenging, since the planner must decide where to search in an infinite branching hierarchy. We propose Graph Sparse Sampling (GSS), an online planning algorithm that shares sampled futures across many candidate decisions, rather than sampling separate successors for each candidate action. This branch-free graph exposes large GPU-friendly batches, while using heuristics to focus computation. We prove finite-sample performance guarantees for GSS covering full-rank or low-rank generative simulators via smoothed backups, and discrete or sampled continuous action spaces. Under suitable overlap, regularity, and action-coverage conditions, these bounds have polynomial dependence on the planning horizon, formalizing when shared futures can avoid the exponential horizon dependence of tree-shaped sparse sampling. We demonstrate continuous-control simulations where GSS substantially outperforms tree-based planners on long horizons or achieves near-optimal performance, supporting no-branching graph planning as a complementary design principle for online control.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。