用微调的开源模型从车祸描述中精准提取隐含信息,效果优于闭源大模型。
Domain-Adapted Pre-trained Language Models for Implicit Information Extraction in Crash Narratives
- 用低秩适配微调开源模型,注入事故领域知识。
- 在CISS数据集上识别碰撞方式和车辆事故类型准确率超GPT-4o。
- 适合交通安全管理、事故分析等需隐私保护的场景。
真实世界车祸数据库中的自由文本事故描述对提升交通安全具有重要意义。然而,由于文本内容非结构化且作者背景差异大,大规模分析难以实现。近年来,基于Transformer的预训练语言模型(如BERT和大型语言模型LLMs)在自然语言处理任务中表现优异,能提取显性事实,但在需要复杂推理的任务(如包含近100类别的事故类型识别)中性能下降。依赖闭源大模型通过外部API也带来敏感数据隐私风险,且黑箱模型因领域知识有限而表现不佳。为此,本文研究紧凑的开源预训练模型是否可支持事故文本中的推理密集型信息抽取。目标包括:1)识别事故的碰撞方式;2)为每辆涉事车辆确定事故类型。通过低秩适配(LoRA)和BERT微调方法弥补领域差距。在权威数据集Crash Investigation Sampling System(CISS)上的实验表明,微调后的紧凑模型在性能上超越强闭源模型GPT-4o,且仅需极少训练资源。进一步分析显示,该模型能捕捉更丰富的叙事细节,甚至纠正部分标注错误。
原文摘要 · Abstract (English)
Free-text crash narratives recorded in real-world crash databases have been shown to play a significant role in improving traffic safety. However, large-scale analyses remain difficult to implement as there are no documented tools that can batch process the unstructured, non standardized text content written by various authors with diverse experience and attention to detail. In recent years, Transformer-based pre-trained language models (PLMs), such as Bidirectional Encoder Representations from Transformers (BERT) and large language models (LLMs), have demonstrated strong capabilities across various natural language processing tasks. These models can extract explicit facts from crash narratives, but their performance declines on inference-heavy tasks in, for example, Crash Type identification, which can involve nearly 100 categories. Moreover, relying on closed LLMs through external APIs raises privacy concerns for sensitive crash data. Additionally, these black-box tools often underperform due to limited domain knowledge. Motivated by these challenges, we study whether compact open-source PLMs can support reasoning-intensive extraction from crash narratives. We target two challenging objectives: 1) identifying the Manner of Collision for a crash, and 2) Crash Type for each vehicle involved in the crash event from real-world crash narratives. To bridge domain gaps, we apply fine-tuning techniques to inject task-specific knowledge to LLMs with Low-Rank Adaption (LoRA) and BERT. Experiments on the authoritative real-world dataset Crash Investigation Sampling System (CISS) demonstrate that our fine-tuned compact models outperform strong closed LLMs, such as GPT-4o, while requiring only minimal training resources. Further analysis reveals that the fine-tuned PLMs can capture richer narrative details and even correct some mislabeled annotations in the dataset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。