让自动驾驶相机模拟突破真实轨迹限制，实现3米外视角的高质量渲染。
FreeSim: Toward Free-viewpoint Camera Simulation in Driving Scenes
- 用生成增强模型和数据构建策略，补全偏移视角的图像。
- 在偏离轨迹超过3米时仍能生成高质量图像。
- 适合需要丰富视角数据的自动驾驶仿真系统开发者。
我们提出 FreeSim，一种面向自动驾驶场景的相机模拟方法。FreeSim 重点提升在未记录自车轨迹之外视角的渲染质量。以往方法在这些视角上因训练数据缺失而表现不佳。为解决数据稀缺问题，我们首先提出一种生成增强模型，结合匹配的数据构建策略，可在轻微偏离轨迹的视角上生成高质量图像，条件是该视角的渲染结果已退化。随后，我们设计渐进式重建策略，从轻微偏离轨迹的视角开始，逐步将生成的未记录视角图像融入重建过程，向更远偏移推进。通过这种生成-重建的渐进式流程，FreeSim 实现了在偏离轨迹超过3米的情况下仍能高质量合成视角图像。
原文摘要 · Abstract (English)
We propose FreeSim, a camera simulation method for autonomous driving. FreeSim emphasizes high-quality rendering from viewpoints beyond the recorded ego trajectories. In such viewpoints, previous methods have unacceptable degradation because the training data of these viewpoints is unavailable. To address such data scarcity, we first propose a generative enhancement model with a matched data construction strategy. The resulting model can generate high-quality images in a viewpoint slightly deviated from the recorded trajectories, conditioned on the degraded rendering of this viewpoint. We then propose a progressive reconstruction strategy, which progressively adds generated images of unrecorded views into the reconstruction process, starting from slightly off-trajectory viewpoints and moving progressively farther away. With this progressive generation-reconstruction pipeline, FreeSim supports high-quality off-trajectory view synthesis under large deviations of more than 3 meters.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。