动态调整视频修复输入帧，提升移动端修复质量
Context-Aware Input Orchestration for Video Inpainting
- 根据光流和掩码变化动态选择输入帧比例
- 在快速视觉变化场景下显著提升修复质量
- 适合移动端低资源环境下的视频修复应用
传统神经网络驱动的视频修复方法在移动设备的算力与内存限制下难以生成高质量结果。本文提出一种优化内存使用的新方法，通过改变输入数据的构成。通常视频修复依赖预设的输入帧集（如邻近帧和参考帧），常限定为五帧。本文研究不同输入帧比例对修复质量的影响，通过基于光流和掩码变化动态调整输入帧组成，在包含快速视觉变化的内容中实现显著性能提升。
原文摘要 · Abstract (English)
Traditional neural network-driven inpainting methods struggle to deliver high-quality results within the constraints of mobile device processing power and memory. Our research introduces an innovative approach to optimize memory usage by altering the composition of input data. Typically, video inpainting relies on a predetermined set of input frames, such as neighboring and reference frames, often limited to five-frame sets. Our focus is to examine how varying the proportion of these input frames impacts the quality of the inpainted video. By dynamically adjusting the input frame composition based on optical flow and changes of the mask, we have observed an improvement in various contents including rapid visual context changes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。