arXiv:2505.24873cs.CV2025-05NeurIPS被引 41

去掉文本输入,用对抗噪声训练,6步就完成高效视频去物。

MiniMax-Remover: Taming Bad Noise Helps Video Object Removal

  • 第一阶段移除文本条件,简化模型;第二阶段用对抗训练优化去物质量。
  • 仅需6次采样步骤即达顶尖效果,且无需分类器自由引导(CFG)。
  • 适合追求快速高质视频编辑的开发者与内容创作者。

视频扩散模型的进步推动了视频编辑技术的快速发展,但视频去物任务仍面临幻觉物体和视觉伪影等问题。现有方法常依赖计算成本高的采样过程和分类器自由引导(CFG),导致推理缓慢。为此,我们提出MiniMax-Remover,一种两阶段视频去物新方法。受文本条件不适用于此任务的启发,第一阶段通过移除文本输入与交叉注意力层,简化预训练视频生成模型,获得更轻量高效的架构。第二阶段在人类标注筛选的成功视频上,采用极小极大优化策略进行模型蒸馏:内层最大化识别使去除失败的“坏噪声”,外层最小化则训练模型在恶劣噪声下仍能生成高质量结果。最终,该方法仅需6次采样即可达到当前最优性能,且无需依赖CFG,显著提升推理效率。大量实验验证了其优越性。代码与演示视频见:https://minimax-remover.github.io。

原文摘要 · Abstract (English)

Recent advances in video diffusion models have driven rapid progress in video editing techniques. However, video object removal, a critical subtask of video editing, remains challenging due to issues such as hallucinated objects and visual artifacts. Furthermore, existing methods often rely on computationally expensive sampling procedures and classifier-free guidance (CFG), resulting in slow inference. To address these limitations, we propose MiniMax-Remover, a novel two-stage video object removal approach. Motivated by the observation that text condition is not best suited for this task, we simplify the pretrained video generation model by removing textual input and cross-attention layers, resulting in a more lightweight and efficient model architecture in the first stage. In the second stage, we distilled our remover on successful videos produced by the stage-1 model and curated by human annotators, using a minimax optimization strategy to further improve editing quality and inference speed. Specifically, the inner maximization identifies adversarial input noise ("bad noise") that makes failure removals, while the outer minimization step trains the model to generate high-quality removal results even under such challenging conditions. As a result, our method achieves a state-of-the-art video object removal results with as few as 6 sampling steps and doesn't rely on CFG, significantly improving inference efficiency. Extensive experiments demonstrate the effectiveness and superiority of MiniMax-Remover compared to existing methods. Codes and Videos are available at: https://minimax-remover.github.io.

视频去物扩散模型高效推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。