让语音内容自动匹配幻灯片元素，实现教学视频自动生成
Script-to-Slide Grounding: Grounding Script Sentences to Slide Objects for Automatic Instructional Video Generation
- 用大语言模型将讲话内容定位到幻灯片文字对象
- 在文本对象上达到0.924的F1分数，效果出色
- 适合教育科技、自动化视频制作领域开发者
基于幻灯片的教学视频虽广泛用于教育和学术展示，但将语音内容与幻灯片元素精准对齐的视觉特效添加过程仍高度依赖人工。本文提出并定义了「脚本到幻灯片定位」（Script-to-Slide Grounding, S2SG）任务，旨在自动将讲话句子与对应幻灯片对象关联。作为初步方案，提出「Text-S2SG」方法，利用大语言模型（LLM）完成文本类对象的定位。实验表明该方法在文本对象上取得0.924的F1分数，显著提升自动化水平。本工作首次将原本隐含的幻灯片视频编辑流程形式化为可计算任务，为后续自动化奠定了基础。
原文摘要 · Abstract (English)
While slide-based videos augmented with visual effects are widely utilized in education and research presentations, the video editing process -- particularly applying visual effects to ground spoken content to slide objects -- remains highly labor-intensive. This study aims to develop a system that automatically generates such instructional videos from slides and corresponding scripts. As a foundational step, this paper proposes and formulates Script-to-Slide Grounding (S2SG), defined as the task of grounding script sentences to their corresponding slide objects. Furthermore, as an initial step, we propose ``Text-S2SG,'' a method that utilizes a large language model (LLM) to perform this grounding task for text objects. Our experiments demonstrate that the proposed method achieves high performance (F1-score: 0.924). The contribution of this work is the formalization of a previously implicit slide-based video editing process into a computable task, thereby paving the way for its automation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。