用三阶段框架结合大模型，提升长期动作预测准确率。
Technical Report for Ego4D Long-Term Action Anticipation Challenge 2025
- 分三步：提取视觉特征、识别动作词、生成未来动作序列
- 在Ego4D数据集上取得第一名，超越现有方法
- 适合关注动作预测与大模型融合的开发者
本文提出一种针对Ego4D长期动作预测（LTA）任务的新型三阶段框架。受基础模型进展启发，该方法包含三个阶段：特征提取、动作识别和长期动作预测。首先使用高性能视觉编码器提取视觉特征；随后将特征输入Transformer，预测动词和名词，并引入动词-名词共现矩阵以提升识别精度；最后将预测的动词-名词对转化为文本提示，输入微调后的大型语言模型（LLM），以预测未来的动作序列。该框架在CVPR 2025举办的Ego4D长期动作预测挑战赛中获得第一名，建立了新的最先进水平。代码将公开于https://github.com/CorrineQiu/Ego4D-LTA-Challenge-2025。
原文摘要 · Abstract (English)
In this report, we present a novel three-stage framework developed for the Ego4D Long-Term Action Anticipation (LTA) task. Inspired by recent advances in foundation models, our method consists of three stages: feature extraction, action recognition, and long-term action anticipation. First, visual features are extracted using a high-performance visual encoder. The features are then fed into a Transformer to predict verbs and nouns, with a verb-noun co-occurrence matrix incorporated to enhance recognition accuracy. Finally, the predicted verb-noun pairs are formatted as textual prompts and input into a fine-tuned large language model (LLM) to anticipate future action sequences. Our framework achieves first place in this challenge at CVPR 2025, establishing a new state-of-the-art in long-term action prediction. Our code will be released at https://github.com/CorrineQiu/Ego4D-LTA-Challenge-2025.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。