arXiv:2605.14894cs.CV2026-05

无需掩码,一步生成即可擦除视频字幕。

SEDiT: Mask-Free Video Subtitle Erasure via One-step Diffusion Transformer

论文配图:SEDiT: Mask-Free Video Subtitle Erasure via One-step Diffusion Transformer
图 1 · 摘自论文原文
  • 提出单阶段扩散变换器,直接无掩码擦除字幕。
  • 实验证明一步去噪可靠,理论证明其可行性。
  • 支持长视频流式处理,适合大尺寸高帧率视频

视频扩散模型的突破加速了视频编辑技术的发展。但现有方法依赖预先提取目标区域掩码进行修复,分割精度直接影响生成质量。本文提出SEDiT,一种基于单步扩散变换器的无掩码视频字幕擦除方法。该方法直接实现字幕擦除,避免两阶段处理的次优性。由于字幕移除是局部编辑任务,多数像素不变,分布偏移小,适合在修正流下一步生成。实验验证一步去噪可靠性,并提供形式化理论支持:在局部编辑结构中,条件最优传输映射及其诱导的修正流速度场关于隐变量具有Lipschitz连续性。为解决长期时序一致性问题,采用混合训练策略,偶尔以干净首帧隐变量作为条件,提升时间连贯性,推理时每一段可利用前一段输出。为避免裁剪重插导致的可见接缝,尤其在大幅运动场景中,直接输入原始视频。得益于一步和分块流式推理,方法可高效处理原生1440p、无限长度视频。

原文摘要 · Abstract (English)

Recent breakthroughs in video diffusion models have significantly accelerated the development of video editing techniques. However, existing methods often rely on inpainting video frames based on masked input, which requires extracting the target video mask in advance, and the precision of the segmentation directly affects the quality of the completion. In this paper, we present SEDiT, a novel one-stage video Subtitle Erasure approach via One-step Diffusion Transformer. We introduce a mask-free inference approach that enables direct erasure of the targeted subtitle. The proposed one-stage framework mitigates the sub-optimality inherent in the two-stage processing of prior models. Since subtitle removal is a localized editing task in which most pixels remain unchanged, the underlying distribution shift is minimal, making it well-suited to one-step generation under rectified flow. We empirically validate the reliability of one-step denoising and further provide a formal theoretical justification. Under the localized-editing structure of subtitle removal, the conditional optimal transport (OT) map and its induced rectified flow velocity field are Lipschitz continuous with respect to the latent variable, which underpins the theoretical feasibility of one-step sampling. To address the challenge of long-term temporal consistency, we adopt a hybrid training strategy by occasionally conditioning the model with a clean first-frame latent. This facilitates temporal continuity, allowing each segment during inference to leverage the output of its predecessor. To avoid visible seams caused by cropping and reinserting processed targets, particularly in scenarios involving substantial motion, we feed the original video directly into SEDiT. Thanks to one-step and chunk-wise streaming inference, our method can efficiently handle native 1440p video with infinite length.

视频编辑扩散模型字幕去除无掩码

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。