用大模型分析车祸描述，提升数据准确率，效果优于传统方法。
Improving Crash Data Quality with Large Language Models: Evidence from Secondary Crash Narratives in Kentucky
- 用微调的Transformer模型识别二次车祸，准确率95%。
- 大模型虽召回率高，但推理耗时长达数小时，成本过高。
- 小规模优化版模型兼顾性能与效率，适合实际部署。
本研究评估先进自然语言处理技术在提升车祸数据质量方面的应用，以肯塔基州二次车祸识别为案例。基于2015-2022年16,656条人工审核的车祸描述，其中3,803条确认为二次车祸，比较三类模型：零样本开源大语言模型（LLaMA3:70B、DeepSeek-R1:70B、Qwen3:32B、Gemma3:27B）、微调变压器模型（BERT、DistilBERT、RoBERTa、XLNet、Longformer）及传统逻辑回归基线。模型在2015-2021年数据上训练，测试于2022年1,771条数据。微调变压器表现最优，RoBERTa达最高F1分数（0.90）和准确率（95%）。零样本LLaMA3:70B F1达0.86，但需139分钟推理；逻辑回归基线显著落后（F1:0.66）。部分大模型召回率高（如Gemma3:27B达0.94），但计算成本极高（如DeepSeek-R1:70B达723分钟）。微调模型仅需短暂训练即可秒级完成测试。进一步分析表明，中等规模大模型（如DeepSeek-R1:32B）可媲美大型模型且提速明显，具备优化部署潜力。结果揭示准确率、效率与数据需求间的权衡，微调变压器在肯塔基数据上实现精度与召回平衡。实际部署建议包括本地化隐私保护、集成方法提效及增量处理以支持扩展，提供了一套可复现的高级NLP提升车祸数据质量方案。
原文摘要 · Abstract (English)
This study evaluates advanced natural language processing (NLP) techniques to enhance crash data quality by mining crash narratives, using secondary crash identification in Kentucky as a case study. Drawing from 16,656 manually reviewed narratives from 2015-2022, with 3,803 confirmed secondary crashes, we compare three model classes: zero-shot open-source large language models (LLMs) (LLaMA3:70B, DeepSeek-R1:70B, Qwen3:32B, Gemma3:27B); fine-tuned transformers (BERT, DistilBERT, RoBERTa, XLNet, Longformer); and traditional logistic regression as baseline. Models were calibrated on 2015-2021 data and tested on 1,771 narratives from 2022. Fine-tuned transformers achieved superior performance, with RoBERTa yielding the highest F1-score (0.90) and accuracy (95%). Zero-shot LLaMA3:70B reached a comparable F1 of 0.86 but required 139 minutes of inference; the logistic baseline lagged well behind (F1:0.66). LLMs excelled in recall for some variants (e.g., GEMMA3:27B at 0.94) but incurred high computational costs (up to 723 minutes for DeepSeek-R1:70B), while fine-tuned models processed the test set in seconds after brief training. Further analysis indicated that mid-sized LLMs (e.g., DeepSeek-R1:32B) can rival larger counterparts in performance while reducing runtime, suggesting opportunities for optimized deployments. Results highlight trade-offs between accuracy, efficiency, and data requirements, with fine-tuned transformer models balancing precision and recall effectively on Kentucky data. Practical deployment considerations emphasize privacy-preserving local deployment, ensemble approaches for improved accuracy, and incremental processing for scalability, providing a replicable scheme for enhancing crash-data quality with advanced NLP.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。