通过并行缩放优化，让个性化扩散模型更准地匹配文本提示。
Parallel Rescaling: Rebalancing Consistency Guidance for Personalized Diffusion Models
- 将一致性引导信号分解为平行与正交分量，仅缩放平行部分。
- 在仅有少量参考图时，仍能保持身份一致性和文本对齐性。
- 无需额外数据或标注，适合快速个性化应用。
将扩散模型个性化以适应特定用户或概念仍具挑战性，尤其在参考图像极少的情况下。现有方法如DreamBooth和Textual Inversion常因过拟合导致生成图像与文本提示不一致。尽管直接一致性优化（DCO）通过一致性引导采样部分缓解此问题，但在复杂或风格化提示下仍表现不佳。本文提出一种并行缩放技术用于个性化扩散模型。该方法将一致性引导信号显式分解为相对于无分类器引导（CFG）的平行与正交分量，仅对平行分量进行缩放，从而最小化对CFG的干扰，同时保留主体身份。与以往方法不同，本技术无需额外训练数据或昂贵标注。大量实验表明，相比基线方法，该技术在挑战性风格化提示下仍实现更好的文本对齐性和视觉保真度。结果表明，并行缩放引导在多样化用户输入下具有提升个性化稳定性和准确性的潜力。
原文摘要 · Abstract (English)
Personalizing diffusion models to specific users or concepts remains challenging, particularly when only a few reference images are available. Existing methods such as DreamBooth and Textual Inversion often overfit to limited data, causing misalignment between generated images and text prompts when attempting to balance identity fidelity with prompt adherence. While Direct Consistency Optimization (DCO) with its consistency-guided sampling partially alleviates this issue, it still struggles with complex or stylized prompts. In this paper, we propose a parallel rescaling technique for personalized diffusion models. Our approach explicitly decomposes the consistency guidance signal into parallel and orthogonal components relative to classifier free guidance (CFG). By rescaling the parallel component, we minimize disruptive interference with CFG while preserving the subject's identity. Unlike prior personalization methods, our technique does not require additional training data or expensive annotations. Extensive experiments show improved prompt alignment and visual fidelity compared to baseline methods, even on challenging stylized prompts. These findings highlight the potential of parallel rescaled guidance to yield more stable and accurate personalization for diverse user inputs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。