在树莓派上实现多模态视觉的实时调度,提升边缘设备效率与隐私保护。
Real-Time Multi-Modal Embedded Vision Framework for Object Detection Facial Emotion Recognition and Biometric Identification on Low-Power Edge Platforms
- 根据上下文动态激活检测、识脸、表情模块,降低65%计算负载。
- 物体检测AP达0.861,识脸准确率88%,情绪识别AUC最高0.97。
- 适合低功耗边缘部署,兼顾性能与隐私,适合智能安防场景。
智能监控系统常独立处理目标检测、人脸识别和情绪分析等感知任务,但缺乏统一且自适应的运行时调度器,无法根据上下文触发动态分配计算资源,限制了其在低功耗边缘设备上的整体理解能力与效率。为此,我们提出一个实时多模态视觉框架,将目标检测、主人专属人脸识别和情绪检测集成于统一管道,并部署在Raspberry Pi 5边缘平台。系统核心为自适应调度机制,通过选择性激活YOLOv8n(目标检测)、基于自定义FaceNet的嵌入系统(人脸识别)及DeepFace的CNN(情绪分类),相较持续运行降低65%计算负载。实验表明,目标检测模块平均精度(AP)达0.861,人脸识别准确率达88%,情绪检测具有强区分能力(特定情绪下AUC高达0.97),系统运行速度为5.6帧/秒。研究证明,上下文感知调度是实现在低成本边缘硬件上复杂多模态AI的关键,使智能感知更易普及且更具隐私保护性。
原文摘要 · Abstract (English)
Intelligent surveillance systems often handle perceptual tasks such as object detection, facial recognition, and emotion analysis independently, but they lack a unified, adaptive runtime scheduler that dynamically allocates computational resources based on contextual triggers. This limits their holistic understanding and efficiency on low-power edge devices. To address this, we present a real-time multi-modal vision framework that integrates object detection, owner-specific face recognition, and emotion detection into a unified pipeline deployed on a Raspberry Pi 5 edge platform. The core of our system is an adaptive scheduling mechanism that reduces computational load by 65\% compared to continuous processing by selectively activating modules such as, YOLOv8n for object detection, a custom FaceNet-based embedding system for facial recognition, and DeepFace's CNN for emotion classification. Experimental results demonstrate the system's efficacy, with the object detection module achieving an Average Precision (AP) of 0.861, facial recognition attaining 88\% accuracy, and emotion detection showing strong discriminatory power (AUC up to 0.97 for specific emotions), while operating at 5.6 frames per second. Our work demonstrates that context-aware scheduling is the key to unlocking complex multi-modal AI on cost-effective edge hardware, making intelligent perception more accessible and privacy-preserving.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。