用检索增强生成让大模型更懂增材制造,效果远超直接微调。
Domain Adaptation of Large Language Models for Polymer-Composite Additive Manufacturing Using Retrieval-Augmented Generation and Fine-Tuning
- 通过向量库检索技术文档,增强大模型对专业知识的调用能力。
- 检索增强模型在200个专家问题中准确率提升至75.5%,相关性达90.8%。
- 适合需要高可靠工程问答的科研与工业场景,尤其依赖多源资料的领域。
通用大语言模型在专业工程领域常因缺乏领域知识支撑而生成不可靠回答。本研究探索将基础大模型适配于增材制造(AM)领域的有效策略,以提升专家级问答的准确性、相关性和可用性。AM知识分散于学术文献、厂商文档、技术标准与操作指南等异构来源中,通用模型虽具备强语言能力,却难以有效检索与上下文关联这些专有信息。本文对比三种基于LLaMA-3-8B的配置:(1)预训练基线模型;(2)从向量数据库中检索文档片段的RAG系统;(3)在原始领域文本上微调的模型。使用200道由机械工程专家设计的问题进行评估,结果表明:RAG模型显著优于基线,75.5%的回答更准确,85.2%被整体偏好,90.8%更相关;而仅在5.6%的情况下微调模型表现更好,相关性仅在32.5%时占优。这说明,在缺乏结构化数据的情况下,检索增强是更有效的领域适配路径。
原文摘要 · Abstract (English)
General-purpose large language models (LLMs) often struggle to generate reliable responses in specialized engineering domains due to limited domain grounding and insufficient exposure to structured technical knowledge. This study investigates practical strategies for adapting a foundation LLM to the additive manufacturing (AM) domain in order to improve answer accuracy, relevance, and usability for expert-level question answering. AM knowledge is distributed across heterogeneous sources such as academic literature, manufacturer documentation, technical standards, and procedural guides. Although general LLMs demonstrate strong linguistic capabilities, they frequently fail to retrieve and contextualize such domain-specific information. Two common approaches to address this limitation are domain-specific fine-tuning and retrieval-augmented generation (RAG). We construct a curated AM corpus and evaluate three configurations based on LLaMA-3-8B: (1) the pretrained baseline model, (2) a RAG system that retrieves relevant document chunks from a vector database, and (3) a model fine-tuned on raw domain text. Performance is evaluated using 200 expert-designed AM questions assessed by mechanical engineering experts for accuracy, relevance, and overall preference. Results show that the RAG model consistently outperforms the baseline. Among the 200 questions, 75.5% of RAG responses are judged more accurate, 85.2% are preferred overall, and 90.8% are rated more relevant than baseline responses. In contrast, fine-tuning on raw AM text reduces performance, producing more accurate answers in only 5.6% of cases and more relevant answers in 32.5% of cases. These results indicate that retrieval-augmented approaches provide a more effective pathway for adapting LLMs to specialized engineering domains than naive fine-tuning on unstructured technical data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。