arXiv:2607.05263cs.CV2026-07被引 1

用概率嵌入提升无监督动作分割效果,避免陷入局部最优。

Learning Probabilistic Embeddings for Unsupervised Action Segmentation

论文配图:Learning Probabilistic Embeddings for Unsupervised Action Segmentation
图 1 · 摘自论文原文
  • 用高斯分布建模帧的潜在表示,采样生成概率嵌入
  • 在多个数据集上提升MoF达20.7%,F1-score最高提升19.0%
  • 适合追求更高分割精度的无监督视频分析研究者

本文研究长时未剪辑视频的无监督时间动作分割问题。现有方法采用联合表示学习与聚类范式,利用最优传输(OT)生成伪标签以优化帧嵌入。但这类方法学习确定性嵌入,导致在伪标签估计与嵌入优化间迭代易陷入局部最优。为此,本文提出学习概率嵌入:将嵌入建模为高斯分布,采样后用于伪标签估计。在多个具有挑战性的动作分割数据集上,本方法性能媲美甚至超过当前最优结果。相比确定性嵌入基线,本方法在MoF指标上最高提升20.7%,F1-score提升最高达19.0%。代码已公开于https://github.com/derkbreeze/PEOT。

原文摘要 · Abstract (English)

This paper concerns the problem of unsupervised temporal action segmentation for long, untrimmed videos. Recent successful approaches follow a joint representation learning and clustering paradigm, where optimal transport (OT) is adopted to produce pseudo labels for learning frame representations. These approaches alternate between estimating pseudo labels using OT and optimizing the parameters with gradient descent during training, where OT is used for obtaining the final temporal action segmentation. A major limitation of these works is that they learn a deterministic embedding for frame representations. The iterative procedure between learning deterministic embeddings based on pseudo labels and estimating pseudo labels from the learned embedding can thus get quickly stuck in a local optimum. As an alternative, we thus propose to learn a probabilistic embedding for frame representations. The embeddings are modeled by Gaussian distributions and we sample from the distributions before estimating the pseudo labels. We evaluate our approach on several challenging temporal action segmentation datasets and achieve results comparable to, and in some cases, better than the state of the art. Compared to baselines with deterministic embeddings, our approach improves MoF up to 20.7\% and F1-score up to 19.0\%. Our code is available at https://github.com/derkbreeze/PEOT.

动作分割概率嵌入无监督学习最优传输

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。