用图像先验知识无标注评估点云质量,跨媒体感知更准。
From Images to Point Clouds: An Efficient Solution for Cross-media Blind Quality Assessment without Annotated Training
- 通过图像与点云的特征分布对齐实现跨媒体质量预测
- 在无标注点云上达到优于传统盲评方法的性能
- 适合缺乏点云标注数据的场景,如真实世界三维重建
我们提出一种新型无标注点云质量评估方法——分布加权图像转移点云质量评估(DWIT-PCQA),可利用图像中的丰富先验知识,从新场景中预测点云的感知质量。基于人类视觉系统(HVS)在各类媒体中均是质量判断主体的认知,我们通过神经网络模拟人眼评价标准,并借助图像先验将质量预测能力从图像迁移至点云。具体而言,采用领域自适应(DA)在统一特征空间中对齐图像与点云特征,但因两类媒体中失真表现差异大,对齐难度高。为此,我们推导出将传统DA优化目标分解为以失真为媒介的两个子目标函数。通过网络实现,提出畸变引导的偏差特征对齐,将现有或估计的失真分布融入对抗性DA框架,强化共性失真模式的对齐。此外,提出质量感知特征解耦机制,缓解因偏差失真对齐导致的质量映射破坏。实验表明,本方法在无需点云标注的情况下,性能显著优于通用盲评方法。
原文摘要 · Abstract (English)
We present a novel quality assessment method which can predict the perceptual quality of point clouds from new scenes without available annotations by leveraging the rich prior knowledge in images, called the Distribution-Weighted Image-Transferred Point Cloud Quality Assessment (DWIT-PCQA). Recognizing the human visual system (HVS) as the decision-maker in quality assessment regardless of media types, we can emulate the evaluation criteria for human perception via neural networks and further transfer the capability of quality prediction from images to point clouds by leveraging the prior knowledge in the images. Specifically, domain adaptation (DA) can be leveraged to bridge the images and point clouds by aligning feature distributions of the two media in the same feature space. However, the different manifestations of distortions in images and point clouds make feature alignment a difficult task. To reduce the alignment difficulty and consider the different distortion distribution during alignment, we have derived formulas to decompose the optimization objective of the conventional DA into two suboptimization functions with distortion as a transition. Specifically, through network implementation, we propose the distortion-guided biased feature alignment which integrates existing/estimated distortion distribution into the adversarial DA framework, emphasizing common distortion patterns during feature alignment. Besides, we propose the quality-aware feature disentanglement to mitigate the destruction of the mapping from features to quality during alignment with biased distortions. Experimental results demonstrate that our proposed method exhibits reliable performance compared to general blind PCQA methods without needing point cloud annotations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。