将教学视频自动生成可交互笔记，保留关键信息与结构。
NoteIt: A System Converting Instructional Videos to Interactable Notes Through Multimodal Video Understanding
- 通过多模态理解提取视频的层级结构与关键信息
- 用户研究显示生成笔记在内容完整性和可用性上表现优异
- 适合需要高效学习与个性化笔记的教育场景
用户常为教学视频做笔记，以避免重复观看。现有自动笔记工具生成的内容难以全面保留原视频信息，且无法满足数字使用中多样化的呈现格式与交互需求。本文提出 NoteIt 系统，通过创新的多模态视频理解流水线，自动将教学视频转化为可交互笔记，精准提取视频的层次结构与多模态关键信息。用户可通过界面自定义笔记内容与展示形式。我们进行了技术评估与用户对照实验（N=36），客观指标表现良好，用户反馈积极，验证了该方法的有效性与系统可用性。
原文摘要 · Abstract (English)
Users often take notes for instructional videos to access key knowledge later without revisiting long videos. Automated note generation tools enable users to obtain informative notes efficiently. However, notes generated by existing research or off-the-shelf tools fail to preserve the information conveyed in the original videos comprehensively, nor can they satisfy users' expectations for diverse presentation formats and interactive features when using notes digitally. In this work, we present NoteIt, a system, which automatically converts instructional videos to interactable notes using a novel pipeline that faithfully extracts hierarchical structure and multimodal key information from videos. With NoteIt's interface, users can interact with the system to further customize the content and presentation formats of the notes according to their preferences. We conducted both a technical evaluation and a comparison user study (N=36). The solid performance in objective metrics and the positive user feedback demonstrated the effectiveness of the pipeline and the overall usability of NoteIt. Project website: https://zhaorunning.github.io/NoteIt/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。