通过中间特征反馈提升图像生成的空间控制精度
Heeding the Inner Voice: Aligning ControlNet Training via Intermediate Features Feedback
- 在每一步去噪过程中用轻量卷积探针重建控制信号
- 使生成结果在所有扩散步骤中保持空间一致性，显著提升控制精度
- 适合需要精细图像编辑与可控生成的研究者和开发者
尽管文本到图像扩散模型已取得显著进展，但实现生成结果的精确空间控制仍具挑战。ControlNet引入辅助条件模块以应对该问题，而ControlNet++通过仅在最终去噪步骤应用循环一致性损失进一步优化对齐。然而，该方法忽略了中间生成阶段，限制了其效果。我们提出InnerControl，一种在所有扩散步骤中强制空间一致性的训练策略。该方法训练轻量级卷积探针，从UNet的每个去噪步骤的中间特征中重建输入控制信号（如边缘、深度）。这些探针能有效从高度噪声潜在表示中提取信号，从而为训练提供伪真值控制信号。通过最小化整个扩散过程中预测与目标条件之间的差异，我们的对齐损失提升了控制保真度与生成质量。结合现有的ControlNet++等技术，InnerControl在多种条件生成方法（如边缘、深度）上达到当前最优性能。
原文摘要 · Abstract (English)
Despite significant progress in text-to-image diffusion models, achieving precise spatial control over generated outputs remains challenging. ControlNet addresses this by introducing an auxiliary conditioning module, while ControlNet++ further refines alignment through a cycle consistency loss applied only to the final denoising steps. However, this approach neglects intermediate generation stages, limiting its effectiveness. We propose InnerControl, a training strategy that enforces spatial consistency across all diffusion steps. Our method trains lightweight convolutional probes to reconstruct input control signals (e.g., edges, depth) from intermediate UNet features at every denoising step. These probes efficiently extract signals even from highly noisy latents, enabling pseudo ground truth controls for training. By minimizing the discrepancy between predicted and target conditions throughout the entire diffusion process, our alignment loss improves both control fidelity and generation quality. Combined with established techniques like ControlNet++, InnerControl achieves state-of-the-art performance across diverse conditioning methods (e.g., edges, depth).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。