arXiv:2505.20626cs.CV2025-05被引 1

无需训练即可实现风格多样下的角色一致性生成

ConsiStyle: Style Diversity in Training-Free Consistent T2I Generation

  • 通过锚定图像提取查询与键,非锚定副本提供值,解耦风格与主体特征
  • 在多个风格下保持角色外观一致,生成图像与文本描述高度对齐
  • 适合需要快速生成多风格一致角色的应用场景

在文本到图像生成中,角色一致性是指在不同提示下保持主体外观的同时实现文本对齐。然而,风格与外观常纠缠在一起,现有方法难以在变化的风格提示下同时维持主体特征一致性和文本对齐性。当前主流方法依赖大规模微调或针对每个主体优化,或泛化能力差,或文本对齐不佳。而训练自由方法通常无法在不同风格间保持主体一致性。本文提出首个无需训练的方法,首次实现了跨多种风格下的风格保留与主体一致性。通过将查询和键从锚定图像获取,值则来自未锚定的并行副本,并扩展自注意力机制中的跨图像组件。为避免偏离目标风格,对值矩阵的统计特性进行对齐。大量定性和定量实验表明,该方法有效解耦风格与主体外观,实现多风格下忠实且一致的角色生成。

原文摘要 · Abstract (English)

In text-to-image models, consistent character generation is the task of achieving text alignment while maintaining the subject's appearance across different prompts. However, since style and appearance are often entangled, the existing methods struggle to preserve consistent subject characteristics while adhering to varying style prompts. Current approaches for consistent text-to-image generation typically rely on large-scale fine-tuning on curated image sets or per-subject optimization, which either fail to generalize across prompts or do not align well with textual descriptions. Meanwhile, training-free methods often fail to maintain subject consistency across different styles. In this work, we introduce a training-free method that, for the first time, jointly achieves style preservation and subject consistency across varied styles. The attention matrices are manipulated such that Queries and Keys are obtained from the anchor image(s) that are used to define the subject, while the Values are imported from a parallel copy that is not subject-anchored. Additionally, cross-image components are added to the self-attention mechanism by expanding the Key and Value matrices. To do without shifting from the target style, we align the statistics of the Value matrices. As is demonstrated in a comprehensive battery of qualitative and quantitative experiments, our method effectively decouples style from subject appearance and enables faithful generation of text-aligned images with consistent characters across diverse styles.

文本生成图像角色一致性无训练方法风格解耦

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。