用压缩光场令牌实现高效自适应神经渲染，可按需调节计算量。
CLiFT: Compressive Light-Field Tokens for Compute-Efficient and Adaptive Neural Rendering
- 将多视角图像压缩为少量光场令牌，保留几何与外观信息。
- 在RealEstate10K和DL3DV上实现数据量显著减少，渲染质量接近原方法。
- 支持动态调整令牌数量，适合资源受限的实时渲染场景。
本文提出一种神经渲染方法，将场景表示为“压缩光场令牌（CLiFTs）”，保留丰富的外观与几何信息。CLiFT通过压缩令牌实现计算高效渲染，且可统一网络下动态调整令牌数量以适应不同视图或计算预算。具体地，给定一组图像与相机位姿，多视角编码器对图像进行令牌化；潜在空间中的K均值聚类基于令牌选取射线簇中心作为聚类质心；多视角“压缩器”将所有令牌信息压缩至质心令牌，构建CLiFTs。测试时，给定目标视图与计算预算（即令牌数量），系统选取指定数量的邻近令牌，并通过自适应渲染器合成新视图。在RealEstate10K与DL3DV数据集上的大量实验表明，该方法在保持相近渲染质量的前提下，实现了显著的数据压缩，获得最高综合渲染得分，同时提供数据量、渲染质量与速度之间的灵活权衡。
原文摘要 · Abstract (English)
This paper proposes a neural rendering approach that represents a scene as "compressed light-field tokens (CLiFTs)", retaining rich appearance and geometric information of a scene. CLiFT enables compute-efficient rendering by compressed tokens, while being capable of changing the number of tokens to represent a scene or render a novel view with one trained network. Concretely, given a set of images, multi-view encoder tokenizes the images with the camera poses. Latent-space K-means selects a reduced set of rays as cluster centroids using the tokens. The multi-view ``condenser'' compresses the information of all the tokens into the centroid tokens to construct CLiFTs. At test time, given a target view and a compute budget (i.e., the number of CLiFTs), the system collects the specified number of nearby tokens and synthesizes a novel view using a compute-adaptive renderer. Extensive experiments on RealEstate10K and DL3DV datasets quantitatively and qualitatively validate our approach, achieving significant data reduction with comparable rendering quality and the highest overall rendering score, while providing trade-offs of data size, rendering quality, and rendering speed.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。