轻量级多任务面部行为分析工具，支持实时运行
OpenFace 3.0: A Lightweight Multitask System for Comprehensive Facial Behavior Analysis
- 统一模型联合训练多种面部分析任务
- 在多个数据集上性能优于同类工具
- 单行命令安装，无需专用硬件
近年来，计算机视觉、人机交互、机器人和情感计算等领域对自动面部行为分析系统兴趣日增。基于先前开源系统的广泛适用性，我们推出 OpenFace 3.0，一个开源工具包，可实现面部关键点检测、面部动作单元识别、眼动估计和面部情绪识别。OpenFace 3.0 提供一个轻量级统一模型，通过多任务架构在多样人群、头部姿态、光照条件、视频分辨率及不同分析任务上进行训练。利用统一模型与训练范式带来的参数共享优势，OpenFace 3.0 在预测性能、推理速度和内存效率方面均优于同类工具，并媲美最先进模型。该工具可通过单行代码安装并实时运行，无需专用硬件。OpenFace 3.0 的训练与运行代码免费用于科研，支持社区贡献。
原文摘要 · Abstract (English)
In recent years, there has been increasing interest in automatic facial behavior analysis systems from computing communities such as vision, multimodal interaction, robotics, and affective computing. Building upon the widespread utility of prior open-source facial analysis systems, we introduce OpenFace 3.0, an open-source toolkit capable of facial landmark detection, facial action unit detection, eye-gaze estimation, and facial emotion recognition. OpenFace 3.0 contributes a lightweight unified model for facial analysis, trained with a multi-task architecture across diverse populations, head poses, lighting conditions, video resolutions, and facial analysis tasks. By leveraging the benefits of parameter sharing through a unified model and training paradigm, OpenFace 3.0 exhibits improvements in prediction performance, inference speed, and memory efficiency over similar toolkits and rivals state-of-the-art models. OpenFace 3.0 can be installed and run with a single line of code and operate in real-time without specialized hardware. OpenFace 3.0 code for training models and running the system is freely available for research purposes and supports contributions from the community.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。