提出轻量级孪生网络，提升高分辨率视频质量评估的语义感知能力。
Exploring Simple Siamese Network for High-Resolution Video Quality Assessment
- 共享权重的孪生结构增强技术分支的语义理解
- 在高分辨率数据集上达到当前最佳性能
- 适合关注高分辨率视频评估的研究者
视频质量评估（VQA）中，双分支网络通过独立的技术与美学分支分别衡量低层失真和高层语义感知。然而，现有技术分支依赖从视频中采样的局部小块训练，难以感知高分辨率视频的语义信息。这一缺陷在低分辨率数据上被掩盖，但在高分辨率场景下尤为关键。本文提出SiamVQA，一种简单高效的孪生网络架构，通过共享技术与美学分支的权重，增强技术分支对语义的理解能力，同时引入双交叉注意力层融合两类特征。该方法在高分辨率基准测试中取得领先结果，在低分辨率数据上也表现稳健。代码将开源于：https://github.com/srcn-ivl/SiamVQA。
原文摘要 · Abstract (English)
In the research of video quality assessment (VQA), two-branch network has emerged as a promising solution. It decouples VQA with separate technical and aesthetic branches to measure the perception of low-level distortions and high-level semantics respectively. However, we argue that while technical and aesthetic perspectives are complementary, the technical perspective itself should be measured in semantic-aware manner. We hypothesize that existing technical branch struggles to perceive the semantics of high-resolution videos, as it is trained on local mini-patches sampled from videos. This issue can be hidden by apparently good results on low-resolution videos, but indeed becomes critical for high-resolution VQA. This work introduces SiamVQA, a simple but effective Siamese network for highre-solution VQA. SiamVQA shares weights between technical and aesthetic branches, enhancing the semantic perception ability of technical branch to facilitate technical-quality representation learning. Furthermore, it integrates a dual cross-attention layer for fusing technical and aesthetic features. SiamVQA achieves state-of-the-art accuracy on high-resolution benchmarks, and competitive results on lower-resolution benchmarks. Codes will be available at: https://github.com/srcn-ivl/SiamVQA
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。