用事件相机辅助插帧，低功耗生成高帧率视频
Coupled Video Frame Interpolation and Encoding with Hybrid Event Cameras for Low-Power High-Framerate Video
- 通过事件数据指导插帧，降低传感器运行频率
- 直接估算编码数据（如运动矢量），避免生成中间帧
- 适合移动端低功耗高帧率视频采集场景
每一代移动设备都在追求更高分辨率和帧率的视频捕捉。这种质量提升也带来了更高的功耗和计算需求。本文提出一种方法，以降低移动设备拍摄高质量视频的整体功耗。利用视频帧插值（VFI），传感器可工作在较低帧率，从而减少功耗。结合现代RGB混合事件视觉传感器（EVS），事件数据可用于引导插值，显著提升结果质量。若直接应用，插值方法可能带来高昂的计算开销和大量中间数据。本文提出一种视频编码器，无需显式插帧即可生成高帧率视频码流。该方法直接估计编码数据（尤其是运动矢量），因此可直接生成编码视频文件，无需生成中间帧。
原文摘要 · Abstract (English)
Every generation of mobile devices strives to capture video at higher resolution and frame rate than previous ones. This quality increase also requires additional power and computation to capture and encode high-quality media. We propose a method to reduce the overall power consumption for capturing high-quality videos in mobile devices. Using video frame interpolation (VFI), sensors can be driven at lower frame rate, which reduces sensor power consumption. With modern RGB hybrid event-based vision sensors (EVS), event data can be used to guide the interpolation, leading to results of much higher quality. If applied naively, interpolation methods can be expensive and lead to large amounts of intermediate data before video is encoded. This paper proposes a video encoder that generates a bitstream for high frame rate video without explicit interpolation. The proposed method estimates encoded video data (notably motion vectors) rather than frames. Thus, an encoded video file can be produced directly without explicitly producing intermediate frames.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。