arXiv:2412.19847cs.CVcs.AI2024-12被引 2

用符号化向量表示图像生成因子,实现可解释的可控编辑。

Symbolic Disentangled Representations for Images

  • 将每个生成因子表示为同维向量,通过超维度计算叠加得到图像表示。
  • 在dSprites和CLEVR数据集上实现零假设训练下的完全解耦,无需分布假设。
  • 适合需要可解释性编辑的图像生成与属性控制场景。

解耦表征旨在将数据简化为生成它的若干生成因子。传统方法中,这些因子以潜在空间中的向量形式表示,每个坐标对应一个因子,但高维下难以确定哪个坐标对应目标因子。本文提出ArSyD(符号解耦架构),将每个生成因子表示为与输出表征同维度的向量,图像表征通过这些因子向量的叠加获得,称为符号解耦表征。该方法基于超维度计算(Hyperdimensional Computing),允许对符号向量进行操作。解耦由构造保证,训练无需关于底层分布的额外假设,仅需弱监督重建图像。我们在dSprites和CLEVR数据集上验证了ArSyD,并提供了对学习到的符号解耦表征的全面分析。我们还提出了新的解耦度量,可用于比较不同维度潜在表示的方法。ArSyD实现了对物体属性的可控且可解释的编辑,且属性表示维度与对象表示维度一致。

原文摘要 · Abstract (English)

The idea of disentangled representations is to reduce the data to a set of generative factors that produce it. Typically, such representations are vectors in latent space, where each coordinate corresponds to one of the generative factors. The object can then be modified by changing the value of a particular coordinate, but it is necessary to determine which coordinate corresponds to the desired generative factor -- a difficult task if the vector representation has a high dimension. In this article, we propose ArSyD (Architecture for Symbolic Disentanglement), which represents each generative factor as a vector of the same dimension as the resulting representation. In ArSyD, the object representation is obtained as a superposition of the generative factor vector representations. We call such a representation a \textit{symbolic disentangled representation}. We use the principles of Hyperdimensional Computing (also known as Vector Symbolic Architectures), where symbols are represented as hypervectors, allowing vector operations on them. Disentanglement is achieved by construction, no additional assumptions about the underlying distributions are made during training, and the model is only trained to reconstruct images in a weakly supervised manner. We study ArSyD on the dSprites and CLEVR datasets and provide a comprehensive analysis of the learned symbolic disentangled representations. We also propose new disentanglement metrics that allow comparison of methods using latent representations of different dimensions. ArSyD allows to edit the object properties in a controlled and interpretable way, and the dimensionality of the object property representation coincides with the dimensionality of the object representation itself.

图像生成解耦表征符号计算

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。