用渲染对比机制解决新视角合成中的姿态误差问题。
ReCoSplat: Autoregressive Feed-Forward Gaussian Splatting Using Render-and-Compare
- 引入渲染对比模块，用预测视角重建图补偿姿态误差。
- 在100+帧长序列上，KV缓存压缩超90%且保持高质量重建。
- 适合需要鲁棒姿态估计的动态场景新视角生成任务。
在线新视角合成仍具挑战性，需从顺序、常无姿态的观测中实现稳健场景重建。本文提出ReCoSplat，一种支持有/无姿态或相机内参输入的自回归前馈高斯点云模型。尽管基于相机位姿组装局部高斯分布比在全局空间预测更高效，但训练时使用真值位姿会导致推理时预测位姿产生的分布偏差。为此，我们设计了渲染-对比（ReCo）模块：从预测视角渲染当前重建，并与输入观测对比，提供稳定条件信号以弥补姿态误差。为支持长序列，提出混合键值缓存压缩策略，结合早期层截断与块级选择性保留，使100+帧序列的KV缓存规模减少超过90%。ReCoSplat在不同输入设置下于分布内/外基准上均达当前最优性能。代码与预训练模型将公开，项目页面见https://freemancheng.com/ReCoSplat。
原文摘要 · Abstract (English)
Online novel view synthesis remains challenging, requiring robust scene reconstruction from sequential, often unposed, observations. We present ReCoSplat, an autoregressive feed-forward Gaussian Splatting model supporting posed or unposed inputs, with or without camera intrinsics. While assembling local Gaussians using camera poses scales better than canonical-space prediction, it creates a dilemma during training: using ground-truth poses ensures stability but causes a distribution mismatch when predicted poses are used at inference. To address this, we introduce a Render-and-Compare (ReCo) module. ReCo renders the current reconstruction from the predicted viewpoint and compares it with the incoming observation, providing a stable conditioning signal that compensates for pose errors. To support long sequences, we propose a hybrid KV cache compression strategy combining early-layer truncation with chunk-level selective retention, reducing the KV cache size by over 90% for 100+ frames. ReCoSplat achieves state-of-the-art performance across different input settings on both in- and out-of-distribution benchmarks. Code and pretrained models will be released. Our project page is at https://freemancheng.com/ReCoSplat .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。