arXiv:2410.07600cs.CV2024-10被引 2

RNA让视频编辑更简单高效,支持复杂运动场景

RNA: Video Editing with ROI-based Neural Atlas

  • 基于兴趣区域设计新框架,无需分离前景物体
  • 提出专用掩码优化方法,解决遮挡问题
  • 适合普通用户在多动场景下快速高质量编辑

随着以视频为主的社交网络平台兴起,普通用户对视频编辑的需求日益增长。然而,由于相机移动和动态物体等时间变化因素,视频编辑仍具挑战性。现有基于图谱的编辑方法虽能应对部分问题,但在处理复杂运动或多物体场景时表现不佳,且计算开销过大,即使简单编辑也效率低下。本文提出一种基于感兴趣区域(ROI)的新型视频编辑框架——神经图谱(RNA)。与以往方法不同,RNA允许用户直接指定编辑区域,避免了前景分割和前景图谱建模的需要,简化操作流程。但该简化带来新挑战:如何在不依赖额外分割模型的前提下,获取能有效处理由移动物体引起的编辑区域遮挡的掩码。为此,我们提出一种专为该问题设计的掩码优化方法。此外,引入软神经图谱模型用于视频重建,确保编辑结果质量。大量实验表明,RNA提供了更实用高效的编辑方案,在更广泛视频类型上实现优于以往方法的质量表现。

原文摘要 · Abstract (English)

With the recent growth of video-based Social Network Service (SNS) platforms, the demand for video editing among common users has increased. However, video editing can be challenging due to the temporally-varying factors such as camera movement and moving objects. While modern atlas-based video editing methods have addressed these issues, they often fail to edit videos including complex motion or multiple moving objects, and demand excessive computational cost, even for very simple edits. In this paper, we propose a novel region-of-interest (ROI)-based video editing framework: ROI-based Neural Atlas (RNA). Unlike prior work, RNA allows users to specify editing regions, simplifying the editing process by removing the need for foreground separation and atlas modeling for foreground objects. However, this simplification presents a unique challenge: acquiring a mask that effectively handles occlusions in the edited area caused by moving objects, without relying on an additional segmentation model. To tackle this, we propose a novel mask refinement approach designed for this specific challenge. Moreover, we introduce a soft neural atlas model for video reconstruction to ensure high-quality editing results. Extensive experiments show that RNA offers a more practical and efficient editing solution, applicable to a wider range of videos with superior quality compared to prior methods.

视频编辑神经图谱目标区域遮挡处理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。