用视觉变压器精准识别六类车辆,助力骑行安全研究
An Open-Source Two-Stage Computer Vision Pipeline for Fine-Grained Vehicle Classification using Vision Transformers

- 两阶段流程:先用RT-DETR定位车辆,再用微调ViT分类车身类型
- 在密歇根州数据上准确率达94%,跨域测试仍保持89%
- 引入置信度阈值机制,避免错误预测,适合真实道路场景
车辆车身类型是超车事故中骑手受伤严重程度的重要决定因素,但现有文献中尚无可用于自然道路视频的自动化车辆细粒度分类工具。标准目标检测基准仅提供粗略类别(轿车、卡车、巴士、摩托车),而现有细粒度识别系统多基于受控图像训练,缺乏在不同采集站点部署鲁棒性的评估。本文提出一个开源的两阶段计算机视觉流水线:首先使用预训练的RT-DETR检测器进行粗粒度车辆定位,随后采用微调的Vision Transformer(ViT-Base/16)对六类车身类型进行分类:乘用车、SUV、皮卡、小型厢式车、大型厢式车和商用卡车。通过置信度阈值机制(低于0.60时放弃预测),生成未知标签而非错误分类。在密歇根州安娜堡自行车道走廊的3,805个标注超车事件上评估,准确率达0.94,各类别F1得分介于0.91(小型厢式车)至0.97(SUV)之间。在未重新训练的独立外部数据集(311个事件)上,准确率为0.89,其中四种主要类别在域偏移下仍有至少0.90的F1得分。小型厢式车性能下降最显著(F1=0.72),主要由拒判率从2.4%升至25.0%导致,反映模型真实不确定性,符合设计预期。完整流水线(含推理脚本、训练代码、评估工具及模型权重)已开源,支持道路视频档案与骑行安全研究的可复现与重用。
原文摘要 · Abstract (English)
Vehicle body type is a significant determinant of cyclist injury severity in overtaking crashes, yet automated tools for classifying vehicles into injury-risk-relevant categories from naturalistic roadway video do not exist in the open literature. Standard object detection benchmarks provide only coarse vehicle labels (car, truck, bus, motorcycle), while existing fine-grained recognition systems are trained on controlled imagery and lack evaluation for deployment robustness across recording sites. This paper presents an open-source two-stage computer vision pipeline combining a pre-trained RT-DETR detector for coarse vehicle localization with a fine-tuned Vision Transformer (ViT-Base/16) for six-category body-type classification: passenger car, SUV, pickup truck, minivan, large van, and commercial truck. A confidence-based abstention mechanism withholds Stage 2 predictions when softmax output falls below 0.60, producing unknown labels rather than silent misclassifications. Evaluated on 3,805 annotated overtaking events from a bicycle-lane corridor in Ann Arbor, Michigan (in-distribution), the pipeline achieved 0.94 accuracy with per-class F1 scores from 0.91 (minivan) to 0.97 (SUV). On an independent out-of-distribution evaluation of 311 events from an open cycling dataset without retraining, accuracy was 0.89. Three of four well-represented categories maintained F1 at or above 0.90 under domain shift. The largest degradation was observed for minivan (F1 = 0.72), driven by abstention rate rising from 2.4% to 25.0% rather than active misclassification, consistent with the mechanism propagating genuine model uncertainty. The full pipeline, including inference scripts, training code, evaluation utilities, and model weights, is released as open-source software to support reproducibility and reuse across roadside video archives and cycling safety research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。