提出折射场模型，让虚拟视角合成能穿透有畸变的保护罩。
Through the Curved Cover: Synthesizing Cover Aberrated Scenes with Refractive Field
- 用折射场建模保护罩几何，精确计算光线折射路径。
- 在真实与合成场景中，渲染质量显著优于已有方法。
- 支持多种曲面罩体，适合扩展现实与机器人视觉应用。
近期增强现实头显和野外机器人广泛采用保护罩来防护前置摄像头免受环境危害与跌落损伤。但保护罩表面不规则会导致模糊和非参数化畸变等光学像差，现有新视角合成方法如NeRF和3D Gaussian Splatting难以处理此类问题。为此，本文提出SynthCover，实现通过保护罩的精准新视角合成，以支持下游扩展现实应用。SynthCover引入折射场（Refractive Field），估计覆盖物几何结构，从而实现折射光线的精确解析计算。在合成与真实场景上的实验表明，该方法能准确建模透过保护罩所见场景，在渲染质量上显著优于先前方法。同时，模型对不同曲率的保护罩具有良好的适应性，可在多种曲面覆盖条件下生成高质量图像。为推动该方向研究，本文还构建了包含真实与合成可步行场景、带有保护罩光学像差的基准数据集。
原文摘要 · Abstract (English)
Recent extended reality headsets and field robots have adopted covers to protect the front-facing cameras from environmental hazards and falls. The surface irregularities on the cover can lead to optical aberrations like blurring and non-parametric distortions. Novel view synthesis methods like NeRF and 3D Gaussian Splatting are ill-equipped to synthesize from sequences with optical aberrations. To address this challenge, we introduce SynthCover to enable novel view synthesis through protective covers for downstream extended reality applications. SynthCover employs a Refractive Field that estimates the cover's geometry, enabling precise analytical calculation of refracted rays. Experiments on synthetic and real-world scenes demonstrate our method's ability to accurately model scenes viewed through protective covers, achieving a significant improvement in rendering quality compared to prior methods. We also show that the model can adjust well to various cover geometries with synthetic sequences captured with covers of different surface curvatures. To motivate further studies on this problem, we provide the benchmarked dataset containing real and synthetic walkable scenes captured with protective cover optical aberrations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。