针对非对齐参考图的图像质量评估，提升新视角合成效果判断能力。
Non-Aligned Reference Image Quality Assessment for Novel View Synthesis
- 基于对比学习框架，融合LoRA增强的DINOv2特征，适配非对齐参考图。
- 在合成失真数据上训练，实现对齐与非对齐参考下均超越现有方法。
- 首次开展人类主观偏好研究，验证模型预测与真实感知高度一致。
新视角合成（NVS）图像的感知质量评估仍是一大挑战，尤其在缺乏像素级对齐参考图的情况下。全参考（FR-IQA）方法因错位失效，无参考（NR-IQA）方法泛化能力不足。本文提出一种专为NVS设计的非对齐参考（NAR-IQA）框架，假设参考视图共享部分场景内容但无像素对齐。我们构建了大规模合成失真数据集，聚焦时间兴趣区域（TROI），用于训练该模型。模型采用对比学习架构，结合LoRA增强的DINOv2嵌入，并由现有IQA方法提供监督信号。训练仅使用合成失真，避免对特定真实NVS样本过拟合，从而提升泛化能力。所提模型在对齐与非对齐参考下均优于先进FR-IQA、NR-IQA及NAR-IQA方法。此外，我们开展新颖用户研究，收集非对齐参考下人类观看偏好数据，发现模型预测与主观评分强相关。项目主页：https://stootaghaj.github.io/nova-project/
原文摘要 · Abstract (English)
Evaluating the perceptual quality of Novel View Synthesis (NVS) images remains a key challenge, particularly in the absence of pixel-aligned ground truth references. Full-Reference Image Quality Assessment (FR-IQA) methods fail under misalignment, while No-Reference (NR-IQA) methods struggle with generalization. In this work, we introduce a Non-Aligned Reference (NAR-IQA) framework tailored for NVS, where it is assumed that the reference view shares partial scene content but lacks pixel-level alignment. We constructed a large-scale image dataset containing synthetic distortions targeting Temporal Regions of Interest (TROI) to train our NAR-IQA model. Our model is built on a contrastive learning framework that incorporates LoRA-enhanced DINOv2 embeddings and is guided by supervision from existing IQA methods. We train exclusively on synthetically generated distortions, deliberately avoiding overfitting to specific real NVS samples and thereby enhancing the model's generalization capability. Our model outperforms state-of-the-art FR-IQA, NR-IQA, and NAR-IQA methods, achieving robust performance on both aligned and non-aligned references. We also conducted a novel user study to gather data on human preferences when viewing non-aligned references in NVS. We find strong correlation between our proposed quality prediction model and the collected subjective ratings. For dataset and code, please visit our project page: https://stootaghaj.github.io/nova-project/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。