arXiv:2410.05564cs.LGcs.CV2024-10TPAMI被引 3

通过稀疏变换分解学习序列数据的解耦表示,无需标签也能捕捉关键变化规律。

Unsupervised Representation Learning from Sparse Transformation Analysis

  • 将潜在变量变换分解为稀疏的旋转与势流场,仅激活少量核心变化模式。
  • 在序列数据上实现最优数据似然与最低近似等变误差,优于现有方法。
  • 适合研究无监督表示学习、物理规律建模或动态系统分析的学者。

现有表示学习多基于编码效率、统计独立性、因果性、可控性或对称性等原则。本文提出从序列数据中通过因子分解潜在变量的变换来学习表示:输入数据首先被编码为潜在激活分布,再经概率流模型变换后解码预测未来状态。该流模型被分解为多个旋转变场(无散)和势流场(无旋),通过稀疏先验约束任意时刻仅少数场活跃,并推断概率沿这些场的流动速度。整个模型完全无监督训练,采用标准变分目标,得到一种新型解耦表示——输入不仅由独立因素构成,还由独立的变换基元(即学习到的流场)组合而成。若将变换视为对称性,可理解为学习近似等变表示。实验表明,该模型在序列变换数据集上实现了最佳的数据似然与最低的无监督近似等变误差。

原文摘要 · Abstract (English)

There is a vast literature on representation learning based on principles such as coding efficiency, statistical independence, causality, controllability, or symmetry. In this paper we propose to learn representations from sequence data by factorizing the transformations of the latent variables into sparse components. Input data are first encoded as distributions of latent activations and subsequently transformed using a probability flow model, before being decoded to predict a future input state. The flow model is decomposed into a number of rotational (divergence-free) vector fields and a number of potential flow (curl-free) fields. Our sparsity prior encourages only a small number of these fields to be active at any instant and infers the speed with which the probability flows along these fields. Training this model is completely unsupervised using a standard variational objective and results in a new form of disentangled representations where the input is not only represented by a combination of independent factors, but also by a combination of independent transformation primitives given by the learned flow fields. When viewing the transformations as symmetries one may interpret this as learning approximately equivariant representations. Empirically we demonstrate that this model achieves state of the art in terms of both data likelihood and unsupervised approximate equivariance errors on datasets composed of sequence transformations.

表示学习无监督稀疏变换等变表示

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。