用大模型增强工业设备维护的语义理解,提升故障诊断效率。
Towards Building General Purpose Embedding Models for Industry 4.0 Agents
- 用LLM扩增查询描述,生成更上下文相关的嵌入表示。
- 在多任务数据集上,检索准确率提升超50%(HIT@1+54.2%)。
- 可支持复杂问题推理与工具调用,适合工业专家日常决策。
本文聚焦于提升语言模型对工业4.0领域资产维护的理解能力,以辅助工程师决策并减少设备停机时间。针对自然语言表达的任务及关联特定资产的查询,目标是推荐相关项并泛化至相似资产的查询。任务可能包括根据设备故障模式识别相关传感器。研究首先构建了九个由专家验证的资产专用任务数据集,并利用大语言模型(LLMs)对输入任务进行扩增,提供查询中涉及实体的简洁描述,以生成更具上下文感知的嵌入表示。该嵌入模型被集成至推理与执行代理(ReAct)中,用于回答需多步推理、规划和知识推断的复杂用户问题。消融实验表明:(a) LLM查询扩增显著提升嵌入质量;(b) 对包含大量候选项的数据集,对比损失及其他避免批内负样本的方法表现更优;(c) 批内正负样本需平衡。在自建数据集上训练与测试后,平均各项指标显著提升:HIT@1提高54.2%,MAP@100提升50.1%,NDCG@10提升54.7%。此外,实证展示了模型在回答工业资产维护复杂问题时的规划与工具调用能力,验证其在支持领域专家日常操作中的有效性。
原文摘要 · Abstract (English)
In this work we focus on improving language models' understanding for asset maintenance to guide the engineer's decisions and minimize asset downtime. Given a set of tasks expressed in natural language for Industry 4.0 domain, each associated with queries related to a specific asset, we want to recommend relevant items and generalize to queries of similar assets. A task may involve identifying relevant sensors given a query about an asset's failure mode. Our approach begins with gathering a qualitative, expert-vetted knowledge base to construct nine asset-specific task datasets. To create more contextually informed embeddings, we augment the input tasks using Large Language Models (LLMs), providing concise descriptions of the entities involved in the queries. This embedding model is then integrated with a Reasoning and Acting agent (ReAct), which serves as a powerful tool for answering complex user queries that require multi-step reasoning, planning, and knowledge inference. Through ablation studies, we demonstrate that: (a) LLM query augmentation improves the quality of embeddings, (b) Contrastive loss and other methods that avoid in-batch negatives are superior for datasets with queries related to many items, and (c) It is crucial to balance positive and negative in-batch samples. After training and testing on our dataset, we observe a substantial improvement: HIT@1 increases by +54.2%, MAP@100 by +50.1%, and NDCG@10 by +54.7%, averaged across all tasks and models. Additionally, we empirically demonstrate the model's planning and tool invocation capabilities when answering complex questions related to industrial asset maintenance, showcasing its effectiveness in supporting Subject Matter Experts (SMEs) in their day-to-day operations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。