arXiv:2605.15585cs.AIcs.CV2026-05

让AI先看图再写代码,生成更流畅的教育动画

See Before You Code: Learning Visual Priors for Spatially Aware Educational Animation Generation

论文配图:See Before You Code: Learning Visual Priors for Spatially Aware Educational Animation Generation
图 1 · 摘自论文原文
  • 用视觉规划模块预判关键帧布局,避免元素重叠错位
  • 在真实渲染后检测缺陷,修复率提升37%以上
  • 适合需要高质量教学动画的开发者和教育科技团队

大语言模型可生成教育动画代码,但渲染结果常出现元素重叠、对齐错误和动画中断等视觉缺陷。这些缺陷无法仅从代码中发现,必须在执行后才能识别。本文将此问题定义为‘渲染反馈感知的约束代码生成’:根据自然语言描述生成代码,使其渲染输出满足可执行的结构化质量标准。为此,提出OmniManim框架,包含共享场景状态、显式视觉规划、结构化后渲染诊断和局部修复机制。其中,视觉代理(Vision Agent)是任务专用的视觉规划模块,通过粗到细的边界框去噪预测稀疏关键帧布局,并优化插值感知目标,减少因下游动画插值导致的中间帧失败。构建了两个数据集ManimLayout-1K和EduRequire-500,提供可复现的评估协议,涵盖可执行性、教学质量、视觉质量和效率。在EduRequire-500上,OmniManim在渲染质量上优于单模型基线和现有多智能体框架。系统性消融实验验证了显式视觉规划——尤其是粗空间先验、边界框精炼与插值感知优化——是性能提升的核心。

原文摘要 · Abstract (English)

Large language models can generate executable code for educational animations, but the resulting renders often exhibit visual defects, including element overlap, misalignment, and broken animation continuity. These defects cannot be reliably detected from the code alone and become apparent only after execution. We formalize this problem as render-feedback-aware constrained code generation: given a natural language specification, the model must generate executable code whose rendered output satisfies structured quality criteria that can be evaluated only after rendering. To address this problem, we introduce OmniManim, a render-feedback-aware educational animation generation framework built around a shared scene state, explicit visual planning, structured post-render diagnostics, and localized repair. Within OmniManim, the Vision Agent is a task-specific visual planning module: it predicts sparse keyframe layouts with coarse-to-fine bounding-box denoising and optimizes an interpolation-aware objective to reduce intermediate-frame failures induced by downstream animation interpolation. We further construct two datasets, ManimLayout-1K and EduRequire-500, and provide a reproducible evaluation protocol covering executability, instructional quality, visual quality, and efficiency. On EduRequire-500, OmniManim improves measured render quality over both single-model baselines and existing multi-agent frameworks. Systematic ablation studies further verify that explicit visual planning, especially its coarse spatial prior, bounding-box refinement, and interpolation-aware optimization, is central to these gains.

教育动画视觉规划代码生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。