用大模型+虚拟现实，让工人远程获专家指导
Bridging Industrial Expertise and XR with LLM-Powered Conversational Agents
- 大模型结合检索增强生成，实现工业知识自然语言交互
- 语义分块与高效向量库提升知识检索准确率
- 适合制造业培训、远程维护等场景，支持无接触操作
本文提出一种将检索增强生成（RAG）增强的大语言模型（LLM）与扩展现实（XR）技术融合的新系统，以解决工业环境中知识传递难题。该系统通过自然语言接口将领域特定的工业知识嵌入XR环境，为工人提供免手持、上下文感知的专家指导。系统架构包含具备动态工具调度能力的LLM聊天引擎和基于语音交互的XR应用。对多种分块策略、嵌入模型及向量数据库的性能评估表明，语义分块、平衡型嵌入模型与高效向量存储可实现最优工业知识检索效果。早期在机器人装配、智能基础设施维护及航空航天部件服务等多类工业场景中的实施验证了其潜力，结果表明该系统有助于提升培训效率、远程协助能力与操作指导水平，契合工业5.0以人为本、韧性强的发展理念。
原文摘要 · Abstract (English)
This paper introduces a novel integration of Retrieval-Augmented Generation (RAG) enhanced Large Language Models (LLMs) with Extended Reality (XR) technologies to address knowledge transfer challenges in industrial environments. The proposed system embeds domain-specific industrial knowledge into XR environments through a natural language interface, enabling hands-free, context-aware expert guidance for workers. We present the architecture of the proposed system consisting of an LLM Chat Engine with dynamic tool orchestration and an XR application featuring voice-driven interaction. Performance evaluation of various chunking strategies, embedding models, and vector databases reveals that semantic chunking, balanced embedding models, and efficient vector stores deliver optimal performance for industrial knowledge retrieval. The system's potential is demonstrated through early implementation in multiple industrial use cases, including robotic assembly, smart infrastructure maintenance, and aerospace component servicing. Results indicate potential for enhancing training efficiency, remote assistance capabilities, and operational guidance in alignment with Industry 5.0's human-centric and resilient approach to industrial development.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。