arXiv:2510.05532cs.CVcs.GR2025-10SIGGRAPH被引 3

用多个模型协作扩展输入输出通道,无需修改原模型结构。

Teamwork: Collaborative Diffusion with Low-rank Coordination and Adaptation

  • 通过多实例协同与低秩适配实现通道扩展
  • 支持动态启停模型实例,灵活适应新任务
  • 在图像修复、材质估计等任务中表现优异

大型预训练扩散模型在图形生成应用中具有强大先验。然而,神经渲染、SVBRDF估计和固有图像分解等生成与逆向任务需要额外的输入或输出通道。现有通道扩展方法通常依赖特定应用,难以适配不同扩散模型或新任务。本文提出Teamwork:一种灵活高效的统一方案,可联合扩展输入输出通道并适配预训练扩散模型至新任务。Teamwork通过协调与适配多个基础扩散模型实例(即队友)实现通道扩展,不改变原有模型架构。采用一种新型低秩适配(LoRA)方法,同时解决队友间的适配与协调问题。此外,Teamwork支持队友的动态激活与去激活。我们在多种生成与逆向图形任务中验证其灵活性与效率,包括图像修复、单图SVBRDF估计、固有分解、神经着色与固有图像合成。

原文摘要 · Abstract (English)

Large pretrained diffusion models can provide strong priors beneficial for many graphics applications. However, generative applications such as neural rendering and inverse methods such as SVBRDF estimation and intrinsic image decomposition require additional input or output channels. Current solutions for channel expansion are often application specific and these solutions can be difficult to adapt to different diffusion models or new tasks. This paper introduces Teamwork: a flexible and efficient unified solution for jointly increasing the number of input and output channels as well as adapting a pretrained diffusion model to new tasks. Teamwork achieves channel expansion without altering the pretrained diffusion model architecture by coordinating and adapting multiple instances of the base diffusion model (\ie, teammates). We employ a novel variation of Low Rank-Adaptation (LoRA) to jointly address both adaptation and coordination between the different teammates. Furthermore Teamwork supports dynamic (de)activation of teammates. We demonstrate the flexibility and efficiency of Teamwork on a variety of generative and inverse graphics tasks such as inpainting, single image SVBRDF estimation, intrinsic decomposition, neural shading, and intrinsic image synthesis.

扩散模型图像修复协同生成低秩适配

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。