arXiv:2606.30296cs.AI2026-06

让AI动画生成器学会跨任务自我改进,靠的是自己积累的成功与失败经验。

ManimAgent: Self-Evolving Multimodal Agents for Visual Education

论文配图:ManimAgent: Self-Evolving Multimodal Agents for Visual Education
图 1 · 摘自论文原文
  • 用双通道记忆库存下每次生成的成败经验,不更新模型参数。
  • 记忆越多,成功率越高,反思次数越少,人类评分通过率超基准30%。
  • 适合教育类多模态工具研发者,尤其关注自动动画生成的场景。

多轮反思使基于大语言模型的智能体能在单个任务内纠正错误,但每个任务仍被视为孤立事件:前一任务中积累的反思经验在下一任务开始时被丢弃。我们在此研究该空白,聚焦于从科学论文段落生成用于数学动画的Python代码任务(使用开源Manim库)。提出ManimAgent,一种自演化多模态智能体,通过完全由自身任务流构建的双通道情景记忆库,实现跨任务的经验传承,无需权重更新,也无需人工标注。每次动画生成完成后,视觉-语言模型对关键帧进行评分;得分信号分别填入正向通道M+(存储成功理由为软参考示例)和负向通道M-(存储经验证的失败模式为硬已知陷阱)。在固定探针评估中,相比无记忆、同预算检索增强生成及打乱记忆基线,随着记忆库增大,盲评人类通过率(Pass@1)提升,反思轮次下降。代码、冻结记忆快照及任务流将公开发布。

原文摘要 · Abstract (English)

Multi-round reflection lets agents built on large language models recover from failures within a single task, but each task remains an isolated episode: lessons learned across many reflection rounds on one task are discarded before the next begins. We study this gap on a code-generation task: from a scientific paper section, the agent writes Python in the open-source Manim library to render a mathematical animation. We present ManimAgent, a self-evolving multimodal agent that carries reflection experience across tasks through a dual-channel Episodic Memory Bank grown entirely from its own task stream, with no weight updates and no human seeds. After each animation converges, a vision-language model scores the rendered keyframes; the resulting signals populate a positive channel M+ that stores success rationales as soft Reference Examples, and a negative channel M- that stores validated failure patterns as hard Known Pitfalls. On a fixed-probe evaluation against no-memory, matched-budget retrieval-augmented generation, and shuffled-memory baselines, blind human Pass@1 rises and reflection rounds fall as memory size grows. We will release the code, frozen memory snapshots, and the task stream.

多模态自演化教育科技动画生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。