首个面向半导体领域的专用大模型，可精准理解蚀刻等工艺难题。
SemiKong: Curating, Training, and Evaluating A Semiconductor Industry-Specific Large Language Model
- 构建半导体领域专有语料库，涵盖制造与设计文本
- 微调后性能超越更大通用模型，解决蚀刻问题更准确
- 提供专家知识融合框架，适合企业定制化模型研发
大型语言模型（LLMs）在半导体行业展现潜力，但普遍缺乏该领域特有的专业知识，难以应对器件与工艺中复杂的物理化学问题。SemiKong 是首个面向半导体领域的专用大模型，旨在建立可支持企业定制模型的基础。通过构建全面的半导体相关文本语料库，并基于此训练具备深度领域知识的预训练模型，我们验证了 SemiKong 1.0 在蚀刻问题理解等制造与设计任务中，优于更大规模的通用模型。此外，我们提出一种融合专家知识的评估框架，推动领域特定 AI 模型的发展。实验表明，开发领域专用基础模型对于后续企业或工具专属模型具有重要意义。代码与数据集将开源于 https://github.com/aitomatic/semikong。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have demonstrated the potential to address some issues within the semiconductor industry. However, they are often general-purpose models that lack the specialized knowledge needed to tackle the unique challenges of this sector, such as the intricate physics and chemistry of semiconductor devices and processes. SemiKong, the first industry-specific LLM for the semiconductor domain, provides a foundation that can be used to develop tailored proprietary models. With SemiKong 1.0, we aim to develop a foundational model capable of understanding etching problems at an expert level. Our key contributions include (a) curating a comprehensive corpus of semiconductor-related texts, (b) creating a foundational model with in-depth semiconductor knowledge, and (c) introducing a framework for integrating expert knowledge, thereby advancing the evaluation process of domain-specific AI models. Through fine-tuning a pre-trained LLM using our curated dataset, we have shown that SemiKong outperforms larger, general-purpose LLMs in various semiconductor manufacturing and design tasks. Our extensive experiments underscore the importance of developing domain-specific LLMs as a foundation for company- or tool-specific proprietary models, paving the way for further research and applications in the semiconductor domain. Code and dataset will be available at https://github.com/aitomatic/semikong
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。