从视频生成以实体为中心的提问，助力学习与问答系统。
ECIS-VQG: Generation of Entity-centric Information-seeking Questions from Videos
- 构建多模态融合模型，结合标题、字幕等上下文信号。
- 在2265条标注问题上达到BLEU 71.3，验证实用价值。
- 首个大规模实体导向视频提问数据集，适合教育与对话系统。
现有视频问答生成研究多聚焦于常见物体和属性，缺乏以实体为中心的提问能力。本文提出针对视频中实体信息的提问生成任务，适用于视频学习、推荐‘相关问题’、视频聊天机器人及事实核查。面对识别关键信息、关联实体、利用多模态信号三大挑战，我们构建了包含411个YouTube视频、2265条人工标注问题的VideoQuestions数据集。提出结合Transformer与多源上下文信号（标题、字幕、字幕嵌入）的模型架构，并采用交叉熵与对比损失联合优化。最佳模型在标准指标上表现优异：BLEU 71.3，ROUGE 78.6，CIDEr 7.31，METEOR 81.9。代码与数据集已公开。
原文摘要 · Abstract (English)
Previous studies on question generation from videos have mostly focused on generating questions about common objects and attributes and hence are not entity-centric. In this work, we focus on the generation of entity-centric information-seeking questions from videos. Such a system could be useful for video-based learning, recommending ``People Also Ask'' questions, video-based chatbots, and fact-checking. Our work addresses three key challenges: identifying question-worthy information, linking it to entities, and effectively utilizing multimodal signals. Further, to the best of our knowledge, there does not exist a large-scale dataset for this task. Most video question generation datasets are on TV shows, movies, or human activities or lack entity-centric information-seeking questions. Hence, we contribute a diverse dataset of YouTube videos, VideoQuestions, consisting of 411 videos with 2265 manually annotated questions. We further propose a model architecture combining Transformers, rich context signals (titles, transcripts, captions, embeddings), and a combination of cross-entropy and contrastive loss function to encourage entity-centric question generation. Our best method yields BLEU, ROUGE, CIDEr, and METEOR scores of 71.3, 78.6, 7.31, and 81.9, respectively, demonstrating practical usability. We make the code and dataset publicly available. https://github.com/thePhukan/ECIS-VQG
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。