arXiv:2506.03828cs.AIcs.MA2025-06KDD被引 17

构建工业资产运维的AI代理评估基准,支持真实场景自动化测试。

AssetOpsBench: Benchmarking AI Agents for Task Automation in Industrial Asset Operations and Maintenance

  • 设计多模态评测框架,整合4类领域专用代理与140+真实工况问题
  • 通过3项指标对比工具型与规划执行型代理架构,发现后者更稳定
  • 提供可复现的仿真环境和开源平台,适合工业智能研究者使用

工业资产全生命周期管理旨在自动化复杂运营流程,如状态监测与维护调度,以减少系统停机时间。传统AI/ML方法仅解决孤立的单一任务,而大语言模型(LLM)代理为端到端自动化提供了新一代机遇。本文提出AssetOpsBench,一个面向工业4.0领域的统一框架,用于编排与评估特定领域的智能体。该框架包含四个领域专用智能体、140多个基于真实工业场景的人类撰写的自然语言查询数据集,以及基于CouchDB的模拟物联网环境。我们引入自动化评估框架,采用三项关键指标分析工具型代理与计划-执行范式之间的架构权衡,并建立系统化方法自动发现新兴故障模式。实际应用价值通过社区广泛采纳得以验证:已有250余名用户和超过500个智能体提交至公开评测平台,支持真实工业运营中的可复现、可扩展研究。代码开源地址:https://github.com/IBM/AssetOpsBench。

原文摘要 · Abstract (English)

AI for Industrial Asset Lifecycle Management aims to automate complex operational workflows, such as condition monitoring and maintenance scheduling, to minimize system downtime. While traditional AI/ML approaches solve narrow tasks in isolation, Large Language Model (LLM) agents offer a next-generation opportunity for end-to-end automation. In this paper, we introduce AssetOpsBench, a unified framework for orchestrating and evaluating domain-specific agents for Industry 4.0. AssetOpsBench provides a multimodal ecosystem comprising a catalog of four domain-specific agents, a curated dataset of 140+ human-authored natural-language queries grounded in real industrial scenarios, and a simulated, CouchDB-backed IoT environment. We introduce an automated evaluation framework that uses three key metrics to analyze architectural trade-offs between the Tool-As-Agent and Plan-Executor paradigms, along with a systematic procedure for the automated discovery of emerging failure modes. The practical relevance of AssetOpsBench is demonstrated by its broad community adoption, with 250+ users and over 500 agents submitted to our public benchmarking platform, supporting reproducible and scalable research for real-world industrial operations. The code is accesible at https://github.com/IBM/AssetOpsBench .

工业AI智能体评估自动化运维大模型应用

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。