用视觉反馈迭代优化物理仿真代码生成,解决语法正确但物理错误的问题。
Perceptual Self-Reflection in Agentic Physics Simulation Code Generation
- 通过视觉语言模型分析动画帧实现代码自我校正
- 7个物理领域测试中多数达目标精度阈值
- 每条动画生成成本约0.2美元,适合工程与科研场景
我们提出一种多智能体框架,从自然语言描述生成物理仿真代码,引入新型感知自省机制进行验证。系统包含四个专用智能体:自然语言解释器将用户请求转为物理描述,技术参数生成器输出缩放后的仿真参数,具备自动纠错能力的物理代码生成器,以及实现感知自省的物理验证器。核心创新在于感知验证:使用具备视觉能力的语言模型分析渲染的动画帧,而非直接检查代码结构。该方法解决了传统测试无法发现的“预言机差距”——即语法正确但物理行为错误的问题。我们在经典力学、流体动力学、热力学、电磁学、波动力学、反应-扩散系统及非物理数据可视化共七个领域评估系统。感知自省架构显著优于单次生成基线,在多数测试场景中达到目标物理精度阈值。系统表现出稳健的流程稳定性,具备持续代码自纠正能力,平均每条动画生成成本约为0.20美元。结果验证了将视觉仿真输出反馈至视觉语言模型以实现迭代优化的假设,表明代理式AI在支持工程工作流和物理数据生成方面具有潜力。
原文摘要 · Abstract (English)
We present a multi-agent framework for generating physics simulation code from natural language descriptions, featuring a novel perceptual self-reflection mechanism for validation. The system employs four specialized agents: a natural language interpreter that converts user requests into physics-based descriptions; a technical requirements generator that produces scaled simulation parameters; a physics code generator with automated self-correction; and a physics validator that implements perceptual self-reflection. The key innovation is perceptual validation, which analyzes rendered animation frames using a vision-capable language model rather than inspecting code structure directly. This approach addresses the ``oracle gap'' where syntactically correct code produces physically incorrect behavior--a limitation that conventional testing cannot detect. We evaluate the system across seven domains including classical mechanics, fluid dynamics, thermodynamics, electromagnetics, wave physics, reaction-diffusion systems, and non-physics data visualization. The perceptual self-reflection architecture demonstrates substantial improvement over single-shot generation baselines, with the majority of tested scenarios achieving target physics accuracy thresholds. The system exhibits robust pipeline stability with consistent code self-correction capability, operating at approximately \$0.20 per animation. These results validate our hypothesis that feeding visual simulation outputs back to a vision-language model for iterative refinement significantly outperforms single-shot code generation for physics simulation tasks and highlights the potential of agentic AI to support engineering workflows and physics data generation pipelines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。