用对抗式多智能体框架提升科学计算代码生成的可靠性。
AI-for-Science Low-code Platform with Bayesian Adversarial Multi-Agent Framework
- 三智能体协同:任务管理、代码生成与评估,动态优化测试用例。
- 通过贝叶斯更新提升代码质量,减少错误传播,提升科学任务可靠性。
- 低代码平台,非编程人员也能高效参与科学计算任务。
大型语言模型在自动化科学代码生成方面展现出潜力,但在可靠性、多智能体工作流中的错误传播以及缺乏明确成功指标的领域评估方面仍面临挑战。本文提出一种专为人工智能赋能科学(AI4S)任务设计的贝叶斯对抗多智能体框架,并以低代码平台(LCP)形式实现。三个基于LLM的智能体在贝叶斯框架下协同:任务管理器将用户输入转化为可执行计划与自适应测试用例,代码生成器产生候选方案,评估器提供全面反馈。通过对抗循环,任务管理器迭代优化测试用例以挑战代码生成器,同时结合功能正确性、结构一致性与静态分析等代码质量指标,动态更新提示分布。该测试与代码的联合优化降低了对大模型可靠性的依赖,缓解了科学任务固有的评估不确定性。LCP还通过将非专家提示转化为领域特定需求,简化人机协作,避免了无编码背景从业者手动调优提示的需要。基准测试表明,该平台能生成鲁棒代码并有效抑制错误传播;在地球科学跨学科任务中亦表现出强可靠性,优于现有模型。
原文摘要 · Abstract (English)
Large Language Models (LLMs) demonstrate potentials for automating scientific code generation but face challenges in reliability, error propagation in multi-agent workflows, and evaluation in domains with ill-defined success metrics. We present a Bayesian adversarial multi-agent framework specifically designed for AI for Science (AI4S) tasks in the form of a Low-code Platform (LCP). Three LLM-based agents are coordinated under the Bayesian framework: a Task Manager that structures user inputs into actionable plans and adaptive test cases, a Code Generator that produces candidate solutions, and an Evaluator providing comprehensive feedback. The framework employs an adversarial loop where the Task Manager iteratively refines test cases to challenge the Code Generator, while prompt distributions are dynamically updated using Bayesian principles by integrating code quality metrics: functional correctness, structural alignment, and static analysis. This co-optimization of tests and code reduces dependence on LLM reliability and addresses evaluation uncertainty inherent to scientific tasks. LCP also streamlines human-AI collaboration by translating non-expert prompts into domain-specific requirements, bypassing the need for manual prompt engineering by practitioners without coding backgrounds. Benchmark evaluations demonstrate LCP's effectiveness in generating robust code while minimizing error propagation. The proposed platform is also tested on an Earth Science cross-disciplinary task and demonstrates strong reliability, outperforming competing models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。