让文本嵌入在球面上操作,实现精准图像编辑。
HEART: Hyperspherical Embedding Alignment via Kent-Representation Traversal in Diffusion Models

- 将文本嵌入视为球面分布,用柯克特分布建模语义结构。
- 无需微调即可实现主体替换与属性调整,保持背景一致。
- 适用于多种扩散模型,适合需要快速可控生成的场景。
文生图扩散模型能生成视觉惊艳的图像,但仅依赖文本条件控制内容和呈现方式仍极困难,例如更换主体或调整属性常导致背景改变或细节失真。这是因为现有方法将嵌入空间视为欧氏空间并使用简单线性变换,未能反映语义概念的真实组织方式。本文探究嵌入空间的真实几何结构,发现文本编码器表示位于超球面上,语义概念并非线性方向,而是由柯克特分布更好地刻画的非各向同性分布。基于此,提出无需训练的HEART框架,在超球面上执行柯克特感知的测地线变换。该方法尊重底层几何结构,实现了直观且精确的编辑,如一致的主体替换与细粒度属性控制,同时保留原始场景。重要的是,HEART无需微调、反演或优化,且在不同扩散模型架构间具有良好泛化性。结果表明,从线性到球面的视角转变,可实现快速且可控的图像生成。
原文摘要 · Abstract (English)
Text-to-image diffusion models can generate visually stunning images, yet, controlling what appears and how it appears, remains surprisingly difficult, especially when operating solely within the constraints of the text-conditioning space. For example, changing a subject or adjusting an attribute often leads to unintended side effects, such as altered backgrounds or distorted details. This is because most existing text-based control methods treat the embedding space as Euclidean and apply simple linear transformations, which do not reflect how semantic concepts are actually organized. In this work, we take a step back and ask: what is the true geometry of these embeddings? We find that text encoder representations lie on a hypersphere, where concepts are not linear directions but structured, anisotropic distributions better captured by Kent distributions. Building on this insight, we propose HEART, a training-free framework that performs Kent-aware geodesic transformations directly on the hypersphere. By respecting the underlying geometry, HEART enables intuitive and precise edits, such as consistent subject replacement and fine-grained attribute control, while preserving the original scene. Importantly, HEART requires no finetuning, inversion, or optimization, and generalizes across diffusion model architectures. Our results show that a simple shift in perspective, from linear to spherical, can unlock fast, and controllable image generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。