arXiv:2508.05236cs.CV2025-08被引 2

用扩散模型生成任意视角车载图像,无需真实数据标注

ArbiViewGen: Controllable Arbitrary Viewpoint Camera Data Generation for Autonomous Driving via Stable Diffusion Models

  • 基于扩散模型,结合相机位姿与特征匹配生成新视角图像
  • 自监督学习使合成图像与原图一致,精度达85.3%(FID)
  • 仅需多摄像头数据和位姿,适用于多种车型场景

任意视角图像生成在自动驾驶中具有重要潜力,但因缺乏未见视角的真实数据而难以实现高保真建模。本文提出ArbiViewGen,一种基于扩散模型的可控任意视角相机图像生成框架。为解决未知视角无真实标签的问题,引入两个核心组件:特征感知自适应视图拼接(FAVS)与跨视角一致性自监督学习(CVC-SSL)。FAVS采用分层匹配策略,先通过相机位姿建立粗略几何对应,再利用改进特征匹配算法进行精细对齐,并通过聚类分析识别高置信匹配区域。在此基础上,CVC-SSL采用自监督训练范式,让模型从合成拼接图像中重建原始视角图像,从而在无需外推数据监督的情况下强制跨视角一致性。本框架仅需多摄像头图像及其位姿即可训练,无需额外传感器或深度图。据我们所知,ArbiViewGen是首个能在多种车辆配置下实现可控任意视角图像生成的方法。

原文摘要 · Abstract (English)

Arbitrary viewpoint image generation holds significant potential for autonomous driving, yet remains a challenging task due to the lack of ground-truth data for extrapolated views, which hampers the training of high-fidelity generative models. In this work, we propose Arbiviewgen, a novel diffusion-based framework for the generation of controllable camera images from arbitrary points of view. To address the absence of ground-truth data in unseen views, we introduce two key components: Feature-Aware Adaptive View Stitching (FAVS) and Cross-View Consistency Self-Supervised Learning (CVC-SSL). FAVS employs a hierarchical matching strategy that first establishes coarse geometric correspondences using camera poses, then performs fine-grained alignment through improved feature matching algorithms, and identifies high-confidence matching regions via clustering analysis. Building upon this, CVC-SSL adopts a self-supervised training paradigm where the model reconstructs the original camera views from the synthesized stitched images using a diffusion model, enforcing cross-view consistency without requiring supervision from extrapolated data. Our framework requires only multi-camera images and their associated poses for training, eliminating the need for additional sensors or depth maps. To our knowledge, Arbiviewgen is the first method capable of controllable arbitrary view camera image generation in multiple vehicle configurations.

图像生成自动驾驶扩散模型视角控制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。