为渲染质量优化点云属性压缩,提升视觉体验。
Rendering-Oriented 3D Point Cloud Attribute Compression using Sparse Tensor-based Transformer
- 将点云建模为稀疏张量,设计局部注意力机制的SP-Trans网络。
- 端到端联合压缩与可微渲染,使压缩结果更贴近人眼所见质量。
- 在多视角渲染质量上超越现有方法,适合实时3D可视化场景。
3D可视化技术的发展深刻改变了数字内容的交互方式,其中点云技术因其沉浸式体验而脱颖而出。然而,点云数据量巨大,压缩面临挑战。现有有损点云属性压缩(PCAC)方法主要关注重建误差最小化,但重构后的点云仍需复杂渲染过程,影响最终用户感知质量。本文提出一种端到端深度学习框架——面向渲染的点云属性压缩(RO-PCAC),直接优化多视图渲染图像质量。通过可微渲染机制,显式建模渲染过程对重构点云的影响。同时,将点云表示为稀疏张量,提出基于稀疏张量的Transformer模型SP-Trans,其局部注意力机制适配点云密度分布,有效捕捉点云内部复杂关系,增强特征分析与合成能力。大量实验表明,相比传统、学习型及混合方法,该框架在压缩性能上达到当前最优水平。
原文摘要 · Abstract (English)
The evolution of 3D visualization techniques has fundamentally transformed how we interact with digital content. At the forefront of this change is point cloud technology, offering an immersive experience that surpasses traditional 2D representations. However, the massive data size of point clouds presents significant challenges in data compression. Current methods for lossy point cloud attribute compression (PCAC) generally focus on reconstructing the original point clouds with minimal error. However, for point cloud visualization scenarios, the reconstructed point clouds with distortion still need to undergo a complex rendering process, which affects the final user-perceived quality. In this paper, we propose an end-to-end deep learning framework that seamlessly integrates PCAC with differentiable rendering, denoted as rendering-oriented PCAC (RO-PCAC), directly targeting the quality of rendered multiview images for viewing. In a differentiable manner, the impact of the rendering process on the reconstructed point clouds is taken into account. Moreover, we characterize point clouds as sparse tensors and propose a sparse tensor-based transformer, called SP-Trans. By aligning with the local density of the point cloud and utilizing an enhanced local attention mechanism, SP-Trans captures the intricate relationships within the point cloud, further improving feature analysis and synthesis within the framework. Extensive experiments demonstrate that the proposed RO-PCAC achieves state-of-the-art compression performance, compared to existing reconstruction-oriented methods, including traditional, learning-based, and hybrid methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。