用变异测试法无真值评估多模态行人轨迹预测模型的鲁棒性。
Metamorphic Testing of Multimodal Human Trajectory Prediction
- 设计五种输入变换的变异关系，匹配输出变化预期。
- 通过沃尔德斯坦距离检测概率分布偏移，量化模型异常。
- 适合自动驾驶和机器人系统中轨迹预测模型的可靠性验证。
预测人类轨迹对自动驾驶车辆和移动机器人的安全与可靠性至关重要。然而，由于多模态轨迹预测（HTP）模型通常使用轨迹历史和环境地图等多源输入，并生成随机输出（多个可能未来路径），其严格测试面临挑战，主要在于缺乏明确的测试判据（oracle），因为任何场景下都有多种合理未来路径。本研究提出将变异测试（Metamorphic Testing, MT）作为系统化方法，用于测试多模态HTP系统。针对轨迹预测的复杂性和随机性，构建五种变异关系（MRs），涵盖对历史轨迹数据和语义分割地图的变换：1）保持标签的几何变换（镜像、旋转、缩放），期望输出相应变换；2）改变地图语义标签或引入障碍物，预期轨迹分布发生可预测变化。提出基于概率分布距离度量（如沃尔德斯坦或海林格距离）的统计违规判定标准。研究构建了无需真实轨迹的工具框架，可评估模型在输入变换和环境变化下的鲁棒性。
原文摘要 · Abstract (English)
Context: Predicting human trajectories is crucial for the safety and reliability of autonomous systems, such as automated vehicles and mobile robots. However, rigorously testing the underlying multimodal Human Trajectory Prediction (HTP) models, which typically use multiple input sources (e.g., trajectory history and environment maps) and produce stochastic outputs (multiple possible future paths), presents significant challenges. The primary difficulty lies in the absence of a definitive test oracle, as numerous future trajectories might be plausible for any given scenario. Objectives: This research presents the application of Metamorphic Testing (MT) as a systematic methodology for testing multimodal HTP systems. We address the oracle problem through metamorphic relations (MRs) adapted for the complexities and stochastic nature of HTP. Methods: We present five MRs, targeting transformations of both historical trajectory data and semantic segmentation maps used as an environmental context. These MRs encompass: 1) label-preserving geometric transformations (mirroring, rotation, rescaling) applied to both trajectory and map inputs, where outputs are expected to transform correspondingly. 2) Map-altering transformations (changing semantic class labels, introducing obstacles) with predictable changes in trajectory distributions. We propose probabilistic violation criteria based on distance metrics between probability distributions, such as the Wasserstein or Hellinger distance. Conclusion: This study introduces tool, a MT framework for the oracle-less testing of multimodal, stochastic HTP systems. It allows for assessment of model robustness against input transformations and contextual changes without reliance on ground-truth trajectories.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。