直接用原始拜耳数据+快速运动估计，视频视觉处理提速且精度损失小。
Efficient Bayer-Domain Video Computer Vision with Fast Motion Estimation and Learned Perception Residual
- 跳过传统图像处理，直接用拜耳原始数据输入模型
- 快速并行运动估计算法减少帧间冗余计算
- 轻量残差网络修复运动误差，适合实时视频任务
视频计算机视觉系统面临两大计算挑战：消除无效处理和减少后端推理中的时间冗余，同时保持精度且额外开销最小。为此，我们提出一个联合优化前端与后端的高效视频视觉框架。前端移除传统图像信号处理器（ISP），直接将拜耳原始数据输入拜耳域视觉模型，避免昂贵的人类导向ISP操作。后端引入一种快速且高度并行的运动估计算法，提取帧间时间对应关系以避免冗余计算。为缓解运动不准确导致的伪影，进一步采用轻量级感知残差网络，直接学习感知级残差并优化传播特征。在多个模型和任务上的实验表明，该系统实现显著加速，仅带来轻微性能下降。
原文摘要 · Abstract (English)
Video computer vision systems face substantial computational burdens arising from two fundamental challenges: eliminating unnecessary processing and reducing temporal redundancy in back-end inference while maintaining accuracy with minimal extra computation. To address these issues, we propose an efficient video computer vision framework that jointly optimizes both the front end and back end of the pipeline. On the front end, we remove the traditional image signal processor (ISP) and feed Bayer raw measurements directly into Bayer-domain vision models, avoiding costly human-oriented ISP operations. On the back end, we introduce a fast and highly parallel motion estimation algorithm that extracts inter-frame temporal correspondence to avoid redundant computation. To mitigate artifacts caused by motion inaccuracies, we further employ lightweight perception residual networks that directly learn perception-level residuals and refine the propagated features. Experiments across multiple models and tasks demonstrate that our system achieves substantial acceleration with only minor performance degradation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。