用点云骨架生成逼真驾驶视频,支持闭环模拟与误差稳定。
Point as Skeleton: Accumulated Point Cloud Enhanced Autoregressive Generation for Closed-Loop Autonomous Driving Simulation

- 以点云骨架分离前景背景,提供视角与深度线索
- 在nuScenes和nuPlan上实现高质量闭环生成,误差累积更稳定
- 适合需要真实视觉反馈的自动驾驶仿真研究者
评估端到端自动驾驶仍具挑战性,因现有仿真方法常在闭环交互性(如CARLA)与真实视觉保真度(如nuScenes)间权衡。本文提出「Point as Skeleton」,一种基于状态更新的自回归驾驶视频生成框架,通过逐步更新的本车状态、交通参与者状态、场景地图及点云骨架条件,由自回归生成器合成视觉观测。为支持闭环滚动,引入Reset-and-Roll机制,防止未来信息在推理中跨步传播。为稳定逐步生成中的误差积累,设计点云骨架,将前景与背景解耦,并投影为相机视图下的点绘图与模板深度图,提供外观与几何线索。进一步构建基于nuPlan的渲染级闭环生成接口,用于评估本车偏离原日志时的生成表现。在nuScenes与nuPlan上的实验表明,该方法显著提升闭环滚动中的生成质量,具备高视觉保真的闭环驾驶仿真潜力。代码已开源。
原文摘要 · Abstract (English)
Evaluating end-to-end autonomous driving (E2E-AD) remains challenging, as existing driving simulation methods often trade off closed-loop interactivity (e.g., CARLA) and real-world visual fidelity (e.g., nuScenes). We present \textbf{\emph{Point as Skeleton}}, a generative sensor simulation framework for state-updated autoregressive driving video generation, in which an autoregressive generator synthesizes visual observations from step-wise updated ego states, actor states, scene maps, and point-cloud skeleton conditions. To support closed-loop rollout, we introduce Reset-and-Roll, which adapts rolling diffusion inference to simulation by preventing future-conditioned latent states from being committed across simulation steps. To stabilize error accumulation during step-wise autoregressive rollout, we introduce point-cloud skeletons that decouple foreground and background assets and project them into camera-view painted-point and template-depth conditions, providing appearance and geometric cues. We further implement a nuPlan-based renderer-level closed-loop generative interface for evaluating generation under ego deviations from the original log. Experiments on nuScenes and nuPlan show that \textit{Point as Skeleton} improves autoregressive generation quality during closed-loop rollout, demonstrating its potential for visually faithful closed-loop driving simulation. The code is available at https://github.com/krauwu/point-as-skeleton.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。