用知识图谱分析自动驾驶数据,判断模型是否学到位。
What Did I Learn? Operational Competence Assessment for AI-Based Trajectory Planners
- 把驾驶场景建模为知识图谱,识别未充分训练的复杂场景
- 通过子场景覆盖率和复杂度评估车辆在特定场景下的能力水平
- 适合自动驾驶安全验证与模型可信部署的开发者
自动驾驶系统越来越多地依赖机器学习完成感知与轨迹规划任务,其性能高度依赖训练数据与实际任务的匹配程度。为确保系统可靠运行,必须了解数据集覆盖范围,以评估模型的操作风险。本文提出一种方法,通过将驾驶数据建模为知识图谱,以实体及其关系表示驾驶场景,并查询特定子场景配置在数据集中是否出现。通过分析训练集中子场景配置的覆盖率与复杂度,评估车辆在特定驾驶场景中的操作能力——复杂场景需要更高的覆盖率才能达到高能力水平。该方法应用于NuPlan数据集,构建知识图谱并分析特定驾驶场景的覆盖情况。该框架可有效监控基于该数据集训练的模型能力,对实现可信自动驾驶至关重要。
原文摘要 · Abstract (English)
Automated driving functions increasingly rely on machine learning for tasks like perception and trajectory planning, requiring large, relevant datasets. The performance of these algorithms depends on how closely the training data matches the task. To ensure reliable functioning, it is crucial to know what is included in the dataset to assess the trained model's operational risk. We aim to enhance the safe use of machine learning in automated driving by developing a method to recognize situations that an automated vehicle has not been sufficiently trained on. This method also improves explainability by describing the dataset at a human-understandable level. We propose modeling driving data as knowledge graphs, representing driving scenes with entities and their relationships. These graphs are queried for specific sub-scene configurations to check their occurrence in the dataset. We estimate a vehicle's competence in a driving scene by considering the coverage and complexity of sub-scene configurations in the training set. Higher complexity scenes require greater coverage for high competence. We apply this method to the NuPlan dataset, modeling it with knowledge graphs and analyzing the coverage of specific driving scenes. This approach helps monitor the competence of machine learning models trained on the dataset, which is essential for trustworthy AI to be deployed in automated driving.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。