arXiv:2508.10409cs.ARcs.AI2025-08被引 6

构建用于训练大模型学习模拟电路知识的文本数据集与定制化训练方法。

Dataset Construction for Training LLM to Learn Analog Circuit Knowledge

  • 用多智能体框架将教材拆解为带思考过程的问答四元组。
  • 训练后模型在AMSBench-TQA上达84.59%准确率,提升15.67个百分点。
  • 适合资源受限场景,支持模拟电路设计任务实操。

本文构建了一个用于训练大语言模型(LLM)学习模拟电路知识的文本数据集,并定制了相应的训练技术。数据集通过收集高质量教材,将其分解为细粒度学习节点,再利用多智能体框架构建结构化的问答-思考-解答四元组,涵盖最终答案与推理过程。最终数据集包含726万未标记令牌用于持续预训练(CPT),以及1.1265亿已标记令牌用于监督微调(SFT)。训练策略包括初始模型选择、训练范式、正则化技术及实际实现建议:识别指令模型为合适的初始化起点;确立以SFT为中心的训练范式(因数据分布不均,CPT收益有限);采用KL散度正则化的SFT相较纯SFT提升2.71个百分点。还提供了资源受限环境下的实用训练方案。实验表明,该数据集与训练方法显著增强模型的模拟电路知识。训练后的320亿参数指令模型在AMSBench-TQA基准上达到84.59%准确率,较初始模型提升15.67个百分点,并在Atelier框架下展现出运算放大器设计能力。

原文摘要 · Abstract (English)

This paper constructs a textual dataset for training large language models (LLMs) to learn analog circuit knowledge and customizes LLM training techniques. For dataset construction, high-quality textbooks are collected and decomposed into fine-grained learning nodes, which are then used to construct structured question-thinking-solution-answer (QTSA) quadruples using a multi-agent framework to capture both final answers and thought processes. The resulting dataset consists of 7.26M tokens of unlabeled data for continual pre-training (CPT) and 112.65M tokens of labeled data for supervised fine-tuning (SFT). We customize the training techniques including initial model selection, training paradigms, regularization techniques, and practical implementation references. Instruct models are identified as suitable training initialization points, an SFT-centric training paradigm is established (finding that CPT provides marginal benefits compared with SFT due to imbalanced data distribution), and SFT with KL divergence regularization can achieve a 2.71 percentage-point improvement over SFT alone. A practical training implementation method is provided for resource-constrained scenarios. Experiments demonstrate that the dataset and training techniques enhance LLMs' analog circuit knowledge. The trained 32B instruct model achieves 84.59% accuracy on the AMSBench-TQA benchmark, showing a 15.67 percentage-point improvement over the initial model. The trained model also shows capability in the operational amplifier design task based on the Atelier framework.

模拟电路大模型训练数据集构建指令微调

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。