让视觉语言模型在边缘设备上高效运行，通过动态压缩与自适应计算实现极致瘦身。
Extreme Model Compression for Edge Vision-Language Models: Sparse Temporal Token Fusion and Adaptive Neural Compression
- 动态重用视觉标记，按事件变化触发，减少冗余计算。
- 在保持高精度前提下，模型参数减少2.3倍，计算量降低62倍。
- 适合资源受限的实时边缘场景，如智能摄像头、可穿戴设备。
边缘视觉语言任务对AI模型提出了实时性能要求，需在功耗和内存受限设备上运行。本文提出两种自适应压缩技术——稀疏时间标记融合（STTF）和自适应神经压缩（ANC），结合算法创新与硬件感知优化。与依赖静态剪枝或均匀缩放的传统方法不同，STTF通过事件驱动的变化检测动态复用视觉标记，而ANC则通过学习的路由机制条件性激活编码器分支，实现对场景复杂度的细粒度适应。我们的30亿参数模型TinyGPT-STTF在COCO 2017测试集上达到CIDEr 131.2、BLEU-4 0.38、METEOR 0.31、ROUGE-L 0.56，比LLaVA-1.5 7B高出17.6点CIDEr，参数仅为其2.3倍，设备端浮点运算量减少62倍。TinyGPT-ANC达CIDEr 128.5。在事件视觉任务中，STTF将平均标记数从196降至31（减少84%），准确率仍保持95.6%；ANC在低运动场景中可降低高达90%的计算量。相比强基线，模型精度最高提升4.4%，延迟降低最多13倍。这些成果使高性能视觉语言模型可在真实边缘设备上高效部署。
原文摘要 · Abstract (English)
The demand for edge AI in vision-language tasks requires models that achieve real-time performance on resource-constrained devices with limited power and memory. This paper proposes two adaptive compression techniques -- Sparse Temporal Token Fusion (STTF) and Adaptive Neural Compression (ANC) -- that integrate algorithmic innovations with hardware-aware optimizations. Unlike previous approaches relying on static pruning or uniform scaling, STTF dynamically reuses visual tokens through event-driven change detection, while ANC conditionally activates encoder branches via a learned router, enabling fine-grained adaptation to scene complexity. Our 3B-parameter TinyGPT-STTF achieves CIDEr 131.2, BLEU-4 0.38, METEOR 0.31, and ROUGE-L 0.56 on the COCO 2017 test set, surpassing LLaVA-1.5 7B by 17.6 CIDEr points while using 2.3x fewer parameters and 62x fewer on-device FLOPs. TinyGPT-ANC reaches CIDEr 128.5. On event-based vision tasks, STTF reduces average token count by 84% (from 196 to 31 tokens) while preserving 95.6% accuracy on the DVS128 Gesture dataset, and ANC cuts FLOPs by up to 90% in low-motion scenes. Compared to strong baselines, our models improve accuracy by up to 4.4% and reduce latency by up to 13x. These results enable efficient deployment of capable vision-language models on real-world edge devices.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。