针对压缩视频的3D重建问题,提出新方法提升画面质量和几何一致性。
CompSplat: Compression-aware 3D Gaussian Splatting for Real-world Video
- 通过建模每帧压缩特征,动态加权并自适应剪枝以减少误差累积
- 在严重压缩下,渲染质量与姿态精度显著优于现有方法
- 适合处理长视频、无轨迹信息且被压缩的真实场景重建
从真实世界视频中实现高质量新视角合成(NVS)对文化遗产保护、数字孪生和沉浸式媒体至关重要。然而,真实视频通常包含长序列、不规则相机轨迹和未知位姿,导致重建过程中出现位姿漂移、特征错位和几何畸变。此外,有损压缩会加剧这些问题,引入不一致,逐步降低几何与渲染质量。尽管近期研究已解决长序列或无位姿重建问题,但压缩感知方法仍局限于特定伪影或有限场景,未能充分探索长视频中多样的压缩模式。本文提出 CompSplat,一种显式建模帧级压缩特性的压缩感知训练框架,以缓解帧间不一致与累积几何误差。该方法引入压缩感知帧加权与自适应剪枝策略,显著提升在重压缩条件下的鲁棒性与几何一致性。在 Tanks and Temples、Free、Hike 等挑战性基准上的大量实验表明,CompSplat 在严重压缩条件下实现了当前最优的渲染质量与姿态精度,显著超越多数最新 NVS 方法。
原文摘要 · Abstract (English)
High-quality novel view synthesis (NVS) from real-world videos is crucial for applications such as cultural heritage preservation, digital twins, and immersive media. However, real-world videos typically contain long sequences with irregular camera trajectories and unknown poses, leading to pose drift, feature misalignment, and geometric distortion during reconstruction. Moreover, lossy compression amplifies these issues by introducing inconsistencies that gradually degrade geometry and rendering quality. While recent studies have addressed either long-sequence NVS or unposed reconstruction, compression-aware approaches still focus on specific artifacts or limited scenarios, leaving diverse compression patterns in long videos insufficiently explored. In this paper, we propose CompSplat, a compression-aware training framework that explicitly models frame-wise compression characteristics to mitigate inter-frame inconsistency and accumulated geometric errors. CompSplat incorporates compression-aware frame weighting and an adaptive pruning strategy to enhance robustness and geometric consistency, particularly under heavy compression. Extensive experiments on challenging benchmarks, including Tanks and Temples, Free, and Hike, demonstrate that CompSplat achieves state-of-the-art rendering quality and pose accuracy, significantly surpassing most recent state-of-the-art NVS approaches under severe compression conditions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。