对比轻量CNN与视觉语言模型在联邦视频暴力检测中的能效表现。
Federated Learning for Video Violence Detection: Complementary Roles of Lightweight CNNs and Vision-Language Models for Energy-Efficient Use
- 用轻量3D CNN与微调的VLM进行联邦学习，兼顾隐私与效率。
- 轻量CNN能耗仅240Wh，准确率超90%，比VLM节能58%。
- 适合注重能效与隐私的实时视频监控系统部署。
基于深度学习的视频监控日益需要保护隐私且计算开销低的架构。联邦学习可保护隐私，但部署大型视觉语言模型（VLMs）带来显著能源与可持续性挑战。我们在RWF-2000和RLVS数据集上，针对真实非独立同分布（non-IID）划分，比较了三种联邦暴力检测策略：预训练VLM的零样本推理、基于LoRA微调的LLaVA-NeXT-Video-7B，以及参数为65.8M的3D CNN的个性化联邦学习。所有方法在二分类暴力检测中均超过90%准确率。3D CNN在能量消耗仅240Wh的情况下，校准性能更优（ROC AUC 92.59%），约为联邦LoRA（570Wh）的一半；而VLM提供更丰富的多模态推理能力。通过基于语义相似性和类别排除的分层类别分组，使VLM在UCF-Crime数据集上的多分类准确率从65.31%提升至81%。据我们所知，这是首个对微调的VLM与个性化CNN在联邦暴力检测中的对比模拟研究，包含明确的能耗与碳排放量化。结果支持混合部署策略：日常推理以高效CNN为主，复杂情境下再调用VLM。
原文摘要 · Abstract (English)
Deep learning-based video surveillance increasingly demands privacy-preserving architectures with low computational and environmental overhead. Federated learning preserves privacy but deploying large vision-language models (VLMs) introduces major energy and sustainability challenges. We compare three strategies for federated violence detection under realistic non-IID splits on the RWF-2000 and RLVS datasets: zero-shot inference with pretrained VLMs, LoRA-based fine-tuning of LLaVA-NeXT-Video-7B, and personalized federated learning of a 65.8M-parameter 3D CNN. All methods exceed 90% accuracy in binary violence detection. The 3D CNN achieves superior calibration (ROC AUC 92.59%) at roughly half the energy cost (240 Wh vs. 570 Wh) of federated LoRA, while VLMs provide richer multimodal reasoning. Hierarchical category grouping (based on semantic similarity and class exclusion) boosts VLM multiclass accuracy from 65.31% to 81% on the UCF-Crime dataset. To our knowledge, this is the first comparative simulation study of LoRA-tuned VLMs and personalized CNNs for federated violence detection, with explicit energy and CO2e quantification. Our results inform hybrid deployment strategies that default to efficient CNNs for routine inference and selectively engage VLMs for complex contextual reasoning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。