用物理驱动的可编辑3D点云生成驾驶场景，解决仿真与现实差距问题。
Real2Sim: A Physics-driven and Editable Gaussian Splatting Framework for Autonomous Driving Scenes
- 结合4D高斯点云与可微分物质点法，重建动态驾驶场景
- 支持实例级编辑，实现碰撞等复杂场景的逼真模拟
- 适合自动驾驶感知、规划等任务的数据增强
可靠自动驾驶依赖大规模、高质量标注数据和鲁棒模型。然而，人工采集数据成本高，传统仿真存在持久的现实差距。尽管近期生成框架和辐射场方法提升了视觉保真度，但在时空一致性及物理感知行为方面仍存不足，限制了其在驾驶场景生成中的应用。为此，我们提出 Real2Sim，一个统一框架，将4D高斯点云（4DGS）与可微分物质点法（MPM）求解器结合。Real2Sim 显式将动态驾驶场景重构为时序连续的高斯原语，支持实例级编辑，并模拟真实物体-物体及物体-环境交互。该框架实现了物理感知、高保真的多样化可编辑场景合成，包括碰撞及碰撞后轨迹等挑战性边缘案例。在 Waymo Open Dataset 上的实验验证了 Real2Sim 在渲染、重建、编辑和物理仿真方面的性能，展示了其作为下游任务（如感知、跟踪、轨迹预测、端到端策略学习）数据生成工具的潜力。
原文摘要 · Abstract (English)
Reliable autonomous driving relies on large-scale, well-labeled data and robust models. However, manual data collection is resource-intensive, and traditional simulation suffers from a persistent reality gap. While recent generative frameworks and radiance-field methods improve visual fidelity, they still struggle with temporal and spatial consistency and cannot ensure physics-aware behavior, limiting their applicability to driving scenario generation. To address these challenges, we propose Real2Sim, an unified framework that combines 4D Gaussian Splatting (4DGS) with a differentiable Material Point Method (MPM) solver. Real2Sim explicitly reconstructs dynamic driving scenes as temporally continuous Gaussian primitives, supports instance-level editing, and simulates realistic object-object and object-environment interactions. This framework enables physics-aware, high-fidelity synthesis of diverse, editable scenarios, including challenging corner cases such as collisions and post-impact trajectories. Experiments on the Waymo Open Dataset validate Real2Sim's capabilities in rendering, reconstruction, editing, and physics simulation, demonstrating its potential as a scalable tool for data generation in downstream tasks such as perception, tracking, trajectory prediction, and end-to-end policy learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。