只在必要时捕获彩色帧，大幅降低边缘设备视频感知功耗
Color When It Counts: Grayscale-Guided Online Triggering for Always-On Streaming Video Sensing
- 用连续灰度流保持时间结构，仅在需要时触发彩色捕获
- 仅用8.1%彩色帧即达全彩91.6%性能，显著减少冗余色彩数据
- 无需训练的实时触发机制，适合资源受限的可穿戴设备
始终在线的传感对下一代边缘/可穿戴AI系统至关重要，但持续高保真RGB视频采集对资源受限的移动和边缘平台仍代价过高。我们提出一种高效流式视频理解新范式：始终保留灰度，按需获取彩色。初步研究发现，只要通过连续灰度流保持时间结构，稀疏的RGB帧即可实现相近性能。基于此，我们提出ColorTrigger——一种无需训练的在线触发机制，通过窗口化灰度相似性分析选择性激活彩色捕获。为支持实时边缘部署，ColorTrigger采用轻量级二次规划检测色度冗余，结合信用预算控制与动态令牌路由，协同降低感知与推理成本。在流式视频理解基准测试中，ColorTrigger仅使用8.1%的RGB帧，便达到全彩基线91.6%的性能，证明自然视频中存在显著色度冗余，使资源受限设备实现实用化的始终在线视频感知。
原文摘要 · Abstract (English)
Always-on sensing is essential for next-generation edge/wearable AI systems, yet continuous high-fidelity RGB video capture remains prohibitively expensive for resource-constrained mobile and edge platforms. We present a new paradigm for efficient streaming video understanding: grayscale-always, color-on-demand. Through preliminary studies, we discover that color is not always necessary. Sparse RGB frames suffice for comparable performance when temporal structure is preserved via continuous grayscale streams. Building on this insight, we propose ColorTrigger, an online training-free trigger that selectively activates color capture based on windowed grayscale affinity analysis. Designed for real-time edge deployment, ColorTrigger uses lightweight quadratic programming to detect chromatic redundancy causally, coupled with credit-budgeted control and dynamic token routing to jointly reduce sensing and inference costs. On streaming video understanding benchmarks, ColorTrigger achieves 91.6% of full-color baseline performance while using only 8.1% RGB frames, demonstrating substantial color redundancy in natural videos and enabling practical always-on video sensing on resource-constrained devices.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。