用生成先验与状态空间模型融合预测人脸美感,效果超越现有方法。
SynergyNet: Fusing Generative Priors and State-Space Models for Facial Beauty Prediction
- 双流架构:一个用扩散模型提取局部细节,一个用Mamba捕捉全局结构。
- 在SCUT-FBP5500上达到0.9235的皮尔逊相关系数,创新高。
- 适合关注美学评估、多模态融合或高效视觉建模的研究者。
人脸美感的自动化预测是情感计算中的基准任务,需同时理解局部审美细节(如皮肤纹理)和整体面部和谐(如对称性、比例)。现有基于卷积神经网络(CNN)或视觉变压器(ViT)的模型存在固有架构偏见:CNN擅长局部特征提取,但难以建模长距离依赖;而ViT虽能建模全局关系,但计算成本高昂。本文提出新型双流架构——马布-扩散网络(MD-Net),通过赋予先进模型专一角色来解决这一权衡。第一流采用预训练潜空间扩散模型中的冻结U-Net编码器,提供精细审美特征的强大生成先验;第二流使用视觉马布(Vim),一种现代状态空间模型,以线性时间复杂度高效捕捉全局面部结构。通过交叉注意力机制协同融合互补表征,MD-Net构建出全面且细腻的特征空间用于预测。在SCUT-FBP5500基准测试中,MD-Net达到0.9235的皮尔逊相关系数,创下新纪录,验证了生成模型与序列建模范式融合在复杂视觉评估任务中的巨大潜力。
原文摘要 · Abstract (English)
The automated prediction of facial beauty is a benchmark task in affective computing that requires a sophisticated understanding of both local aesthetic details (e.g., skin texture) and global facial harmony (e.g., symmetry, proportions). Existing models, based on either Convolutional Neural Networks (CNNs) or Vision Transformers (ViTs), exhibit inherent architectural biases that limit their performance; CNNs excel at local feature extraction but struggle with long-range dependencies, while ViTs model global relationships at a significant computational cost. This paper introduces the \textbf{Mamba-Diffusion Network (MD-Net)}, a novel dual-stream architecture that resolves this trade-off by delegating specialized roles to state-of-the-art models. The first stream leverages a frozen U-Net encoder from a pre-trained latent diffusion model, providing a powerful generative prior for fine-grained aesthetic qualities. The second stream employs a Vision Mamba (Vim), a modern state-space model, to efficiently capture global facial structure with linear-time complexity. By synergistically integrating these complementary representations through a cross-attention mechanism, MD-Net creates a holistic and nuanced feature space for prediction. Evaluated on the SCUT-FBP5500 benchmark, MD-Net sets a new state-of-the-art, achieving a Pearson Correlation of \textbf{0.9235} and demonstrating the significant potential of hybrid architectures that fuse generative and sequential modeling paradigms for complex visual assessment tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。