arXiv:2508.10963cs.CV2025-08被引 3

EVCtrl让视觉生成控制更高效,不重训练也能提速两倍以上。

EVCtrl: Efficient Control Adapter for Visual Generation

  • 通过时空双重缓存机制,只在需要处计算控制信号。
  • 在CogVideo和Wan2.1上分别提速2.16倍和2.05倍,质量几乎不变。
  • 无需重训练,可直接插件式接入现有控制生成模型。

视觉生成包括图像与视频生成,通过概率模型从零生成连贯、多样且语义忠实的内容。早期研究聚焦无条件采样,如今用户更需可控生成,能精确指定布局、姿态、运动或风格。尽管ControlNet提供精准时空控制,其辅助分支显著增加延迟,并在非控制区域和去噪步骤中引入冗余计算,尤其在视频生成中更为明显。为此,我们提出EVCtrl——一种轻量级、即插即用的控制适配器,无需重训练即可大幅降低开销。具体而言,我们设计了时空双缓存策略以处理稀疏控制信息:针对空间冗余,先分析DiT-ControlNet各层对细粒度控制的响应,将网络划分为全局与局部功能区;局部感知缓存仅在真正需要控制信号的局部区域计算,跳过全局区域的冗余计算。针对时间冗余,选择性跳过不必要的去噪步骤以提升效率。在CogVideo-Controlnet、Wan2.1-Controlnet和Flux上的大量实验表明,该方法在图像与视频控制生成中均有效,无需重训练。例如,在CogVideo-Controlnet和Wan2.1-Controlnet上分别实现2.16倍和2.05倍的速度提升,生成质量几乎无损。代码见附录。

原文摘要 · Abstract (English)

Visual generation includes both image and video generation, training probabilistic models to create coherent, diverse, and semantically faithful content from scratch. While early research focused on unconditional sampling, practitioners now demand controllable generation that allows precise specification of layout, pose, motion, or style. While ControlNet grants precise spatial-temporal control, its auxiliary branch markedly increases latency and introduces redundant computation in both uncontrolled regions and denoising steps, especially for video. To address this problem, we introduce EVCtrl, a lightweight, plug-and-play control adapter that slashes overhead without retraining the model. Specifically, we propose a spatio-temporal dual caching strategy for sparse control information. For spatial redundancy, we first profile how each layer of DiT-ControlNet responds to fine-grained control, then partition the network into global and local functional zones. A locality-aware cache focuses computation on the local zones that truly need the control signal, skipping the bulk of redundant computation in global regions. For temporal redundancy, we selectively omit unnecessary denoising steps to improve efficiency. Extensive experiments on CogVideo-Controlnet, Wan2.1-Controlnet, and Flux demonstrate that our method is effective in image and video control generation without the need for training. For example, it achieves 2.16 and 2.05 times speedups on CogVideo-Controlnet and Wan2.1-Controlnet, respectively, with almost no degradation in generation quality.Codes are available in the supplementary materials.

视觉生成控制生成加速推理轻量化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。