用双提示机制解耦几何与纹理,提升点云质量评估能力
GT-PCQA: Geometry-Texture Decoupled Point Cloud Quality Assessment with MLLM
- 通过2D-3D联合训练,融合图像与点云数据提升小样本下的指令微调效果
- 在多个基准数据集上达到先进性能,尤其对几何退化更敏感
- 适合需要高精度点云质量评估的三维重建、自动驾驶场景
随着多模态大模型(MLLM)的快速发展,基于MLLM的图像质量评估方法展现出良好泛化能力。然而,将此类方法直接扩展至点云质量评估(PCQA)仍面临挑战:一方面,现有PCQA数据集规模有限,难以支持稳定有效的MLLM指令微调;另一方面,由于大规模图像-文本预训练,MLLM倾向于依赖纹理主导的推理,对点云中关键的几何结构退化不敏感。为此,本文提出一种新型无参考点云质量评估框架GT-PCQA,包含两项关键技术:第一,设计2D-3D联合训练策略,将点云质量评估建模为相对质量比较问题,统一大规模图像质量数据集与有限的点云数据集,并采用参数高效低秩适配(LoRA)方案支持指令微调;第二,提出几何-纹理解耦策略,结合双提示机制与交替优化方案,缓解预训练MLLM固有的纹理偏好,增强对几何结构退化的敏感性。大量实验表明,GT-PCQA表现优异且具备强泛化能力。
原文摘要 · Abstract (English)
With the rapid advancement of Multi-modal Large Language Models (MLLMs), MLLM-based Image Quality Assessment (IQA) methods have shown promising generalization. However, directly extending these MLLM-based IQA methods to PCQA remains challenging. On the one hand, existing PCQA datasets are limited in scale, which hinders stable and effective instruction tuning of MLLMs. On the other hand, due to large-scale image-text pretraining, MLLMs tend to rely on texture-dominant reasoning and are insufficiently sensitive to geometric structural degradations that are critical for PCQA. To address these gaps, we propose a novel MLLM-based no-reference PCQA framework, termed GT-PCQA, which is built upon two key strategies. First, to enable stable and effective instruction tuning under scarce PCQA supervision, a 2D-3D joint training strategy is proposed. This strategy formulates PCQA as a relative quality comparison problem to unify large-scale IQA datasets with limited PCQA datasets. It incorporates a parameter-efficient Low-Rank Adaptation (LoRA) scheme to support instruction tuning. Second, a geometry-texture decoupling strategy is presented, which integrates a dual-prompt mechanism with an alternating optimization scheme to mitigate the inherent texture-dominant bias of pre-trained MLLMs, while enhancing sensitivity to geometric structural degradations. Extensive experiments demonstrate that GT-PCQA achieves competitive performance and exhibits strong generalization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。