用交通工程资料微调大模型,提升专业问答能力
Customized Generative AI Agent for Transportation Engineering Practice: A Development and Continued Pre-training Guideline

- 用美国交通手册微调6个主流大模型,统一通过LoRA框架
- Qwen2.5-7B和LLaMA-3.1-8B在领域对齐上表现最佳
- 为交通工程提供可复现的智能助手开发方案
生成式人工智能和大语言模型在复杂推理、摘要和问答任务中展现出巨大潜力。然而,通用大模型在交通工程等专业领域的应用受限于对技术标准、工程术语和领域语义的缺乏。本研究提出一套系统化方法,构建面向交通工程应用的定制化生成式AI代理。通过整合美国交通手册、设计规范和法规文件,使用统一的低秩适应(LoRA)框架对六个前沿大模型进行持续预训练,并监控训练过程以确保收敛与模型稳定性。性能评估采用标准自然语言处理指标,包括BLEU-4和ROUGE。结果表明,Qwen2.5-7B和LLaMA-3.1-8B在领域对齐和响应质量方面表现最优。研究验证了基于LoRA的适配方法能有效提升大模型在技术内容理解与上下文推理中的表现。本工作贡献了一个可复现的开发框架,支持在交通科研、设计、规划和政策分析中的广泛部署。
原文摘要 · Abstract (English)
Recent advancements in generative artificial intelligence (AI) and large language models (LLMs) have shown significant promise in automating complex reasoning, summarization, and question-answering tasks. However, the effectiveness of general-purpose LLMs in specialized engineering domains remains limited due to insufficient exposure to technical standards, engineering terminology, and domain-specific semantics. This study proposes a systematic approach to developing a customized generative AI agent for transportation engineering applications. A curated corpus of U.S. transportation manuals, design guidelines, and regulatory documents is used to conduct continued pretraining of six state-of-the-art LLMs through a unified low-rank adaptation (LoRA) framework. The training process is monitored to ensure convergence and model stability. Performance is evaluated using standard natural language processing metrics, including BLEU-4 and ROUGE, with Qwen2.5-7B and LLaMA-3.1-8B demonstrating the highest domain alignment and response quality. Results validate the effectiveness of LoRA-based adaptation in improving LLM performance on technical content interpretation and context-specific reasoning. This work contributes a reproducible development framework for constructing domain-specialized generative AI agents, supporting broader deployment in transportation research, design, planning, and policy analysis.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。