arXiv:2409.05396cs.CVcs.AI2024-09被引 10

提出首个可分解面部光流的方法与大规模数据集,提升表情分析精度。

FacialFlowNet: Advancing Facial Optical Flow Estimation with a Diverse Dataset and a Decomposed Model

  • 构建分解式光流模型,分离头部与表情运动成分
  • 在真实场景中使微表情识别准确率提升18%至82.1%
  • 数据集含9635人、10.6万图像对,支持精细化动作分析

面部动作对传达情绪和意图至关重要,面部光流提供了其动态细节表征。然而,因数据集稀缺和现代基线缺失,该领域进展受限。本文提出大型面部光流数据集FacialFlowNet(FFN)及首个可分解面部光流的模型DecFlow。FFN包含9,635个身份和105,970对图像,具备前所未有的多样性,适用于细致的面部与头部运动分析。DecFlow采用面部语义感知编码器与分解式光流解码器,在准确估计并分解面部光流为头部与表情分量方面表现优异。大量实验表明,使用FFN后,各类光流方法的终点误差(EPE)最高降低11%(从3.91降至3.48)。此外,结合FFN时,DecFlow在合成与真实场景中均优于现有方法,显著提升表情分析能力;其分解后的表情光流使微表情识别准确率提升18%(从69.1%升至82.1%)。这些成果推动了面部运动分析与光流估计的重大进展。代码与数据集已公开。

原文摘要 · Abstract (English)

Facial movements play a crucial role in conveying altitude and intentions, and facial optical flow provides a dynamic and detailed representation of it. However, the scarcity of datasets and a modern baseline hinders the progress in facial optical flow research. This paper proposes FacialFlowNet (FFN), a novel large-scale facial optical flow dataset, and the Decomposed Facial Flow Model (DecFlow), the first method capable of decomposing facial flow. FFN comprises 9,635 identities and 105,970 image pairs, offering unprecedented diversity for detailed facial and head motion analysis. DecFlow features a facial semantic-aware encoder and a decomposed flow decoder, excelling in accurately estimating and decomposing facial flow into head and expression components. Comprehensive experiments demonstrate that FFN significantly enhances the accuracy of facial flow estimation across various optical flow methods, achieving up to an 11% reduction in Endpoint Error (EPE) (from 3.91 to 3.48). Moreover, DecFlow, when coupled with FFN, outperforms existing methods in both synthetic and real-world scenarios, enhancing facial expression analysis. The decomposed expression flow achieves a substantial accuracy improvement of 18% (from 69.1% to 82.1%) in micro-expressions recognition. These contributions represent a significant advancement in facial motion analysis and optical flow estimation. Codes and datasets can be found.

面部光流微表情识别分解模型数据集

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。