让扩散模型随生成阶段动态调整控制策略,提升细节精度。
TC-LoRA: Temporally Modulated Conditional LoRA for Adaptive Diffusion Control
- 用超网络实时生成适配器,按时间与条件动态调节模型权重。
- 在多数据集上生成质量显著优于传统静态激活修改方法。
- 适合需要精细空间控制的图像生成任务,如设计、医学成像。
当前可控扩散模型通常依赖固定架构,通过修改中间激活来注入新模态的引导信息。这种静态条件策略难以适应动态的多阶段去噪过程,限制了模型在生成过程中自适应调整响应的能力。本文提出TC-LoRA(时序调制条件LoRA),一种新范式,通过直接条件化模型权重实现动态、上下文感知的控制。该框架利用超网络实时生成LoRA适配器,在每个扩散步骤中根据时间步与用户条件,对冻结主干网络进行定制化权重调整。这一机制使模型能够学习并执行显式的自适应策略,贯穿整个生成过程。在多个数据域上的实验表明,这种参数化、动态控制显著提升了生成保真度和空间条件遵循性。TC-LoRA建立了一种通过深度权重功能适配来修改条件策略的替代路径,使控制更契合任务的动态需求与生成阶段。
原文摘要 · Abstract (English)
Current controllable diffusion models typically rely on fixed architectures that modify intermediate activations to inject guidance conditioned on a new modality. This approach uses a static conditioning strategy for a dynamic, multi-stage denoising process, limiting the model's ability to adapt its response as the generation evolves from coarse structure to fine detail. We introduce TC-LoRA (Temporally Modulated Conditional LoRA), a new paradigm that enables dynamic, context-aware control by conditioning the model's weights directly. Our framework uses a hypernetwork to generate LoRA adapters on-the-fly, tailoring weight modifications for the frozen backbone at each diffusion step based on time and the user's condition. This mechanism enables the model to learn and execute an explicit, adaptive strategy for applying conditional guidance throughout the entire generation process. Through experiments on various data domains, we demonstrate that this dynamic, parametric control significantly enhances generative fidelity and adherence to spatial conditions compared to static, activation-based methods. TC-LoRA establishes an alternative approach in which the model's conditioning strategy is modified through a deeper functional adaptation of its weights, allowing control to align with the dynamic demands of the task and generative stage.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。