模仿创作者流程，用多智能体反馈提升科研短视频生成质量
Stealing Creator's Workflow: A Creator-Inspired Agentic Framework with Iterative Feedback Loop for Improved Scientific Short-form Generation
- 设计多智能体系统，分步完成摘要、场景设计与排版编辑
- 通过用户角色模拟反馈，迭代优化视频生成结果
- 适合需要高质量科研传播的学者与科普创作者
从科学论文生成引人入胜且准确的短视频极具挑战性，源于内容复杂性以及作者与读者之间的认知差距。现有端到端方法常出现事实错误和视觉瑕疵，限制了其在科学传播中的应用。为此，我们提出 SciTalk，一种基于多大模型的智能体框架，将视频生成过程与文本、图表、视觉风格及虚拟形象等多元信息源对齐。受内容创作者工作流程启发，SciTalk 设有专门负责内容摘要、视觉场景规划、文字与布局编辑的智能体，并引入迭代反馈机制：视频智能体模拟用户角色，对前一轮生成的视频提供反馈并优化生成提示。实验表明，相较于简单提示方法，SciTalk 在迭代过程中能生成更科学准确且更具吸引力的内容。尽管初步结果仍不及人类创作者水平，但本框架揭示了反馈驱动视频生成的挑战与优势。代码、数据及生成视频将公开可用。
原文摘要 · Abstract (English)
Generating engaging, accurate short-form videos from scientific papers is challenging due to content complexity and the gap between expert authors and readers. Existing end-to-end methods often suffer from factual inaccuracies and visual artifacts, limiting their utility for scientific dissemination. To address these issues, we propose SciTalk, a novel multi-LLM agentic framework, grounding videos in various sources, such as text, figures, visual styles, and avatars. Inspired by content creators' workflows, SciTalk uses specialized agents for content summarization, visual scene planning, and text and layout editing, and incorporates an iterative feedback mechanism where video agents simulate user roles to give feedback on generated videos from previous iterations and refine generation prompts. Experimental evaluations show that SciTalk outperforms simple prompting methods in generating scientifically accurate and engaging content over the refined loop of video generation. Although preliminary results are still not yet matching human creators' quality, our framework provides valuable insights into the challenges and benefits of feedback-driven video generation. Our code, data, and generated videos will be publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。