用摄像头和麦克风实现语音+人脸双因子认证，兼顾效率与安全
Two-step Authentication: Multi-biometric System Using Voice and Facial Recognition
- 先用人脸识别筛选候选者，再仅对匹配对象进行语音验证，降低计算量
- 人脸识别准确率95.1%，语音验证错误率3.456%，在小样本数据集上表现优异
- 适合移动端部署，开源代码与模型，便于二次开发与实际应用
我们提出一种低成本的双阶段认证系统，仅使用普通设备上的摄像头和麦克风，融合人脸识别与说话人验证。流程首先通过人脸识别从少量注册用户中识别出候选者，再仅对匹配身份进行语音验证，从而减少计算开销并提升鲁棒性。人脸识别采用基于剪枝VGG-16的分类器，在包含5名受试者共924张图像的增强数据集上训练，面部定位由MTCNN完成，准确率达95.1%。语音识别使用在LibriSpeech（train-other-360）上训练的CNN说话人验证模型，在test-clean数据集上达到98.9%准确率和3.456%等错误率（EER）。源代码与训练好的模型已公开于https://github.com/NCUE-EE-AIAL/Two-step-Authentication-Multi-biometric-System。
原文摘要 · Abstract (English)
We present a cost-effective two-step authentication system that integrates face identification and speaker verification using only a camera and microphone available on common devices. The pipeline first performs face recognition to identify a candidate user from a small enrolled group, then performs voice recognition only against the matched identity to reduce computation and improve robustness. For face recognition, a pruned VGG-16 based classifier is trained on an augmented dataset of 924 images from five subjects, with faces localized by MTCNN; it achieves 95.1% accuracy. For voice recognition, a CNN speaker-verification model trained on LibriSpeech (train-other-360) attains 98.9% accuracy and 3.456% EER on test-clean. Source code and trained models are available at https://github.com/NCUE-EE-AIAL/Two-step-Authentication-Multi-biometric-System.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。