基于心理规律的视频美学评估框架,提升模型泛化能力。
Peak-End-Net: A Peak-End Rule Inspired Framework for Generalizable Video Aesthetic Assessment

- 借鉴峰终定律,用关键帧和结尾判断视频整体美感
- 在两个数据集上达到当前最优,跨域测试表现稳健
- 轻量级设计,适合资源有限场景下的美学评估
视频美学评估(VAA)旨在预测视频的审美吸引力,但远不如其他视觉评估任务被充分研究。其进展受限于大规模基准数据稀缺及审美判断的主观性,后者受人类感知影响。本文从心理学视角重新审视VAA,提出基于峰终定律的轻量级可解释框架Peak-End-Net,该定律指出人们对时间体验的评价主要依赖显著时刻和结尾。我们首先通过预训练图像美学评估头迁移知识,生成帧级美学先验,作为识别美学显著帧的代理信号,并指导基于峰终规则的时间聚合。为捕捉视频美学随时间的演变,设计美学节奏编码器,建模超越孤立时刻的时序动态。此外,通过动态门控融合机制优化整体评估,在分布偏移下增强鲁棒性。方法基于冻结的视觉变压器(ViT),仅需少量可训练参数,具备高可扩展性与参数效率。在两个现有基准(VADB域内评估、DIVIDE-3K跨域测试)上的大量实验表明,本方法达到当前最优性能,验证了心理驱动建模对VAA的价值。代码与模型已开源。
原文摘要 · Abstract (English)
Video aesthetic assessment (VAA) aims to predict how aesthetically pleasing a video is, yet remains far less explored than other visual assessment tasks. Its progress is hindered not only by the scarcity of large-scale benchmarks, but also by the intrinsic subjectivity of aesthetic judgment, which is shaped by human perception. In this paper, we revisit VAA from a psychological perspective and propose \textit{Peak-End-Net}, a lightweight and interpretable framework inspired by the \textit{peak-end rule}, which suggests that people tend to judge a temporal experience mainly according to its salient moments and the ending. Building on this intuition, we first transfer knowledge from image aesthetic assessment (IAA) to VAA by introducing a pretrained IAA head to produce frame-wise aesthetic priors, which serve as surrogate signals for identifying aesthetically salient moments and guiding \textit{peak-end rule}-based temporal aggregation. To further capture how a video evolves aesthetically over time, we design an aesthetic rhythm encoder that models temporal progression beyond isolated moments. Additionally, we refine the overall assessment through a dynamic gated fusion mechanism to improve robustness under distribution shift. Our method is built on a frozen vision transformer (ViT) and requires only a small number of trainable parameters, making it scalable and parameter-efficient. Extensive experiments on two existing VAA benchmarks, including in-domain evaluation on VADB and cross-domain testing on DIVIDE-3K, demonstrate that our approach achieves state-of-the-art performance, affirming the value of psychologically grounded modeling for VAA. Our code and models are available at https://github.com/AMAP-ML/Peak-End-Net.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。