用可微方法自动学习任务图，提升第一人称视频动作理解性能。
Task Graph Maximum Likelihood Estimation for Procedural Activity Understanding in Egocentric Videos
- 基于最大似然优化边权重，实现任务图的端到端可微学习。
- 在多个数据集上提升10%以上F1分数，最高达+14.5%。
- 适合做动作理解、错误检测与视频语义解析的研究者参考。
我们提出一种基于梯度的学习方法，用于从程序性活动中学习任务图，优于传统手工设计方法。该方法通过最大似然直接优化边权重，可融入神经网络架构。在CaptainCook4D、EgoPER和EgoProceL数据集上，分别实现+14.5%、+10.2%和+13.6%的F1分数提升。基于特征的任务图预测方法在文本/视频嵌入上展现出初步的视频理解能力。此外，在Ego-Exo4D的流程理解基准测试中取得最佳表现，并显著提升在线错误检测性能（Assembly101-O上+19.8%，EPIC-Tent-O上+6.4%）。代码已开源：https://github.com/fpv-iplab/Differentiable-Task-Graph-Learning。
原文摘要 · Abstract (English)
We introduce a gradient-based approach for learning task graphs from procedural activities, improving over hand-crafted methods. Our method directly optimizes edge weights via maximum likelihood, enabling integration into neural architectures. We validate our approach on CaptainCook4D, EgoPER, and EgoProceL, achieving +14.5%, +10.2%, and +13.6% F1-score improvements. Our feature-based approach for predicting task graphs from textual/video embeddings demonstrates emerging video understanding abilities. We also achieved top performance on the procedure understanding benchmark on Ego-Exo4D and significantly improved online mistake detection (+19.8% on Assembly101-O, +6.4% on EPIC-Tent-O). Code: https://github.com/fpv-iplab/Differentiable-Task-Graph-Learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。