分离点云内容与失真特征,提升无参考质量评估准确性
Learning Disentangled Representations for Perceptual Point Cloud Quality Assessment via Mutual Information Minimization
- 用双分支网络分别学习内容和失真特征,通过最小化互信息实现解耦
- 在多个数据集上超越现有方法,尤其在复杂失真场景下表现更优
- 适合研究点云质量评估或需要解耦表示的3D视觉任务开发者
无参考点云质量评估(NR-PCQA)旨在不依赖原始点云的情况下客观评估点云的人类感知质量,随着虚拟现实(VR)和增强现实(AR)等沉浸式媒体应用的快速发展,其重要性日益凸显。然而,当前的NR-PCQA模型通常在一个网络中混杂学习点云内容与失真表征,忽略了二者对质量信息的不同贡献。为此,我们提出DisPA框架,一种用于NR-PCQA的新型解耦表示学习方法。该框架训练一个双分支解耦网络,通过最小化点云内容与失真表征之间的互信息(MI)来实现解耦。具体而言,为充分解耦表征,两个分支采用不同策略:内容感知编码器通过掩码自编码策略预训练,可从失真点云的渲染图像中捕捉语义信息;失真感知编码器以小块图作为输入,强制其关注低层失真模式。此外,我们使用互信息估计器估算实际互信息的紧致上界,并进一步最小化该值,实现显式的表征解耦。大量实验结果表明,DisPA在多个PCQA数据集上优于现有最先进方法。
原文摘要 · Abstract (English)
No-Reference Point Cloud Quality Assessment (NR-PCQA) aims to objectively assess the human perceptual quality of point clouds without relying on pristine-quality point clouds for reference. It is becoming increasingly significant with the rapid advancement of immersive media applications such as virtual reality (VR) and augmented reality (AR). However, current NR-PCQA models attempt to indiscriminately learn point cloud content and distortion representations within a single network, overlooking their distinct contributions to quality information. To address this issue, we propose DisPA, a novel disentangled representation learning framework for NR-PCQA. The framework trains a dual-branch disentanglement network to minimize mutual information (MI) between representations of point cloud content and distortion. Specifically, to fully disentangle representations, the two branches adopt different philosophies: the content-aware encoder is pretrained by a masked auto-encoding strategy, which can allow the encoder to capture semantic information from rendered images of distorted point clouds; the distortion-aware encoder takes a mini-patch map as input, which forces the encoder to focus on low-level distortion patterns. Furthermore, we utilize an MI estimator to estimate the tight upper bound of the actual MI and further minimize it to achieve explicit representation disentanglement. Extensive experimental results demonstrate that DisPA outperforms state-of-the-art methods on multiple PCQA datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。