用程序匹配真实传感器数据中的故障场景,验证仿真结果是否真实存在。
Querying Labeled Time Series Data with Scenario Programs
- 用Scenic语言定义故障场景程序,精确描述时间序列数据的匹配条件。
- 算法比商用大模型快数个数量级,且能处理更长时序数据。
- 适合自动驾驶安全验证人员,解决仿真与现实之间的差距问题。
基于仿真的测试已成为保障网络物理系统(CPS)安全的重要补充,尤其在自动驾驶领域。然而,一个关键问题是:仿真中发现的车辆故障场景在真实世界中是否可重现?由于仿真与真实传感器数据间的差异,仿真中的故障可能只是合成数据的伪影,而非真实问题。为解决此问题,我们提出一种有效方法:将仿真中发现的故障场景,在真实世界数据集中进行定位,并验证故障是否依然存在。为此,我们正式定义了标注时间序列数据如何匹配抽象场景,使用Scenic概率编程语言表示场景程序。我们设计了一种查询算法,给定场景程序和标注数据集,可快速识别出符合该场景的数据子集。实验表明,该算法在查询精度和速度上均优于当前最先进的商业视觉大模型,且能随查询时间序列长度线性扩展。
原文摘要 · Abstract (English)
Simulation-based testing has become a crucial complement to road testing for ensuring the safety of cyber physical systems (CPS). As a result, significant research efforts have been directed toward identifying failure scenarios within simulation environments. However, a critical question remains. Are the AV failure scenarios discovered in simulation reproducible on actual systems in the real world? The sim-to-real gap caused by differences between simulated and real sensor data means that failure scenarios identified in simulation might either be artifacts of synthetic sensor data or actual issues that also occur with real sensor data. To address this, an effective approach to validating simulated failure scenarios is to locate occurrences of these scenarios within real-world datasets and verify whether the failure persists on the datasets. To this end, we introduce a formal definition of how labeled time series sensor data can match an abstract scenario, represented as a scenario program using the Scenic probabilistic programming language. We present a querying algorithm that, given a scenario program and a labeled dataset, identifies the subset of data that matches the specified scenario. Our experiment shows that our algorithm is more accurate and orders of magnitude faster in querying scenarios than the state-of-the-art commercial vision large language models, and can scale with the duration of queried time series data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。