arXiv:2502.06682cs.CV2025-02CVPR被引 4

让自动驾驶车从任意视角生成真实感知,突破数据采集瓶颈。

Transfer Your Perspective: Controllable 3D Generation from Any Viewpoint in a Driving Scene

  • 用扩散模型基于单视角数据生成多视角真实感知
  • 在无需真实协同数据下实现早期/晚期融合训练
  • 适合自动驾驶协同感知算法研发者快速迭代

依赖自车感知的自动驾驶系统常因遮挡或远距离物体而失效。协同自动驾驶(CAV)虽具前景,但真实场景中需多传感器设备同步部署,数据采集困难,现有数据集在场景和代理数量上均受限。本文提出一种新方法——转移视角(TYP),通过结合仿真协同数据与真实自车数据,构建条件扩散模型,可从真实自车感知数据生成其他视角的逼真感知结果。该方法使任意自车数据集具备协同感知潜力,实验证明其能有效支持早/晚融合等协同感知算法的预训练,显著减少对真实协同数据的依赖,极大促进下游CAV应用发展。

原文摘要 · Abstract (English)

Self-driving cars relying solely on ego-centric perception face limitations in sensing, often failing to detect occluded, faraway objects. Collaborative autonomous driving (CAV) seems like a promising direction, but collecting data for development is non-trivial. It requires placing multiple sensor-equipped agents in a real-world driving scene, simultaneously! As such, existing datasets are limited in locations and agents. We introduce a novel surrogate to the rescue, which is to generate realistic perception from different viewpoints in a driving scene, conditioned on a real-world sample - the ego-car's sensory data. This surrogate has huge potential: it could potentially turn any ego-car dataset into a collaborative driving one to scale up the development of CAV. We present the very first solution, using a combination of simulated collaborative data and real ego-car data. Our method, Transfer Your Perspective (TYP), learns a conditioned diffusion model whose output samples are not only realistic but also consistent in both semantics and layouts with the given ego-car data. Empirical results demonstrate TYP's effectiveness in aiding in a CAV setting. In particular, TYP enables us to (pre-)train collaborative perception algorithms like early and late fusion with little or no real-world collaborative data, greatly facilitating downstream CAV applications.

自动驾驶3D生成扩散模型协同感知

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。