arXiv:2608.05877cs.CVcs.AI2026-08

提出双闭环优化传输方法,提升无监督动作分割精度。

D-CLOT: Double Closed Loop Optimal Transport for Unsupervised Action Segmentation

论文配图:D-CLOT: Double Closed Loop Optimal Transport for Unsupervised Action Segmentation
图 1 · 摘自论文原文
  • 构建双闭环结构,先优化帧嵌入再重估动作原型。
  • 在5个基准上最高提升12.7点F1和10.2点mIoU。
  • 适合需要高精度动作分割的研究者使用。

最优传输(OT)已成为无监督动作分割的有效框架。然而,现有基于OT的方法中,定义OT代价的潜在动作原型并未根据优化后的帧几何进行重估,而是仅通过伪标签损失的梯度演化。我们识别出这一‘表示-原型不一致’是核心瓶颈,尤其在模糊过渡及短时或罕见动作场景下。为此,我们基于近期提出的CLOT方法,进一步在帧嵌入优化后重新估计动作原型。具体地,引入图约束模块,通过保留编码器输出的局部邻域几何来正则化优化后的帧与段嵌入;随后定期将原型锚定至该稳定表示几何。我们提出两种实现:D-CLOT使用k-means更新原型,D-CLOT$_{B}$则以优化后的传输计划加权作为OT重心更新原型,实现与当前传输几何一致的分配感知更新。在五个主流基准上,两种变体均优于CLOT,视频级提升最高达+12.7 F1和+10.2 mIoU(YTI),活动级提升最高+8.9 F1(FS-Eval)。此外,我们在Assembly101上建立首个无监督动作分割基线,该数据集为更精细、流程化的任务。大量消融与敏感性分析表明,两种优化机制具有互补性且鲁棒。

原文摘要 · Abstract (English)

Optimal transport (OT) has emerged as an effective framework for unsupervised action segmentation. Yet, in existing OT-based methods, the latent action prototypes that define the OT costs are not re-estimated from the refined frame geometry. Instead, they evolve solely through gradients from the pseudo-label loss. We identify this \emph{representation--prototype inconsistency} as a central bottleneck, particularly around ambiguous transitions and for short or infrequent actions. To address this issue, we build on the recently introduced CLOT, which refines frame embeddings based on estimated segment embeddings, and further re-estimates the action prototypes from the refined frame embeddings. Specifically, we introduce a graph-constrained module that regularizes the OT-refined frame and segment representations by preserving the local neighborhood geometry of the encoder output. An action-embedding refinement step then periodically re-anchors the prototypes to this stabilized representation geometry. We study two instantiations that share the same backbone, graph module, and objective: D-CLOT updates the prototypes using $k$-means, whereas D-CLOT$_{B}$ updates them as OT barycenters weighted by the refined transport plan, yielding an assignment-aware prototype update consistent with the current transport geometry. Across five established benchmarks, both variants improve segment-level quality over CLOT, with per-video gains of up to $+12.7$ F1 and $+10.2$ mIoU (YTI) and activity-level gains of up to $+8.9$ F1 (FS-Eval). We further establish the first unsupervised action-segmentation baseline on Assembly101, a procedural and substantially more fine-grained benchmark than those commonly used in prior work. Extensive ablations and sensitivity analyses demonstrate that the two refinement mechanisms are complementary and robust.

动作分割最优传输无监督学习图神经网络

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。