用传感器内压缩降低边缘设备能耗，提升能效15.4倍
SnapPix: Efficient-Coding--Inspired In-Sensor Compression for Edge Vision
- 基于高效编码理论设计像素采样模式，实现模拟域压缩
- 在相同速度下，动作识别与视频重建性能优于现有方法
- 适合资源受限的边缘视觉场景，如远程传感与低功耗设备
边缘设备上的节能图像采集对远程传感应用至关重要，尤其当传感器节点计算能力弱且需将数据传至远程服务器处理时。为降低边缘能耗，本文提出一种传感器-算法协同设计系统SnapPix，可在传感器内部模拟域对原始像素进行压缩。采用编码曝光（Coded Exposure, CE）作为传感器内压缩策略，具备时空选择性曝光的灵活性。SnapPix有三大贡献：首先，提出一种与任务无关的采样/曝光模式学习方法，基于高效编码理论；其次，协同设计下游视觉模型以应对CE压缩图像中独特的像素级非均匀性；最后，提出轻量级传感器硬件增强方案以支持该压缩。在动作识别与视频重建任务上评估，SnapPix在相同速率下性能超越当前最优视频方法，同时能耗降低高达15.4倍。代码已开源：https://github.com/horizon-research/SnapPix。
原文摘要 · Abstract (English)
Energy-efficient image acquisition on the edge is crucial for enabling remote sensing applications where the sensor node has weak compute capabilities and must transmit data to a remote server/cloud for processing. To reduce the edge energy consumption, this paper proposes a sensor-algorithm co-designed system called SnapPix, which compresses raw pixels in the analog domain inside the sensor. We use coded exposure (CE) as the in-sensor compression strategy as it offers the flexibility to sample, i.e., selectively expose pixels, both spatially and temporally. SNAPPIX has three contributions. First, we propose a task-agnostic strategy to learn the sampling/exposure pattern based on the classic theory of efficient coding. Second, we co-design the downstream vision model with the exposure pattern to address the pixel-level non-uniformity unique to CE-compressed images. Finally, we propose lightweight augmentations to the image sensor hardware to support our in-sensor CE compression. Evaluating on action recognition and video reconstruction, SnapPix outperforms state-of-the-art video-based methods at the same speed while reducing the energy by up to 15.4x. We have open-sourced the code at: https://github.com/horizon-research/SnapPix.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。