提出新评估框架,检测多智能体轨迹预测中的模式坍缩问题。
Mode Collapse Happens: Evaluating Critical Interactions in Joint Trajectory Prediction Models
- 设计序列化评估指标,量化交互模式的多样性与准确性。
- 实测四类模型均存在模式坍缩,交互前仍无法正确预测行为模式。
- 适用于自动驾驶安全评估,帮助提升预测模型鲁棒性。
自动驾驶决策依赖于考虑多种路径选择和人类行为不确定性的多模态预测模型。然而,模型可能陷入模式坍缩,仅预测最可能的模式,带来重大安全隐患。现有方法虽尝试生成多样化预测,但常忽略智能体间交互模式的多样性。此外,传统评估指标具有数据集依赖性,且无法定量评估智能体间交互。据我们所知,当前尚无指标直接评估模式坍缩。本文提出一种新型评估框架,聚焦安全关键交互,评估联合轨迹预测中的模式坍缩问题。引入模式坍缩、模式正确性与覆盖率等指标,强调预测的时序维度。通过测试四类多智能体轨迹预测模型,我们证实模式坍缩确实发生:尽管在接近交互事件时预测准确率提升,但在交互即将必然发生前,仍有案例未能预测正确的交互模式。我们希望该框架能为研究人员提供新视角,推动更一致、更准确预测模型的发展,从而提升自动驾驶系统安全性。
原文摘要 · Abstract (English)
Autonomous Vehicle decisions rely on multimodal prediction models that account for multiple route options and the inherent uncertainty in human behavior. However, models can suffer from mode collapse, where only the most likely mode is predicted, posing significant safety risks. While existing methods employ various strategies to generate diverse predictions, they often overlook the diversity in interaction modes among agents. Additionally, traditional metrics for evaluating prediction models are dataset-dependent and do not evaluate inter-agent interactions quantitatively. To our knowledge, none of the existing metrics explicitly evaluates mode collapse. In this paper, we propose a novel evaluation framework that assesses mode collapse in joint trajectory predictions, focusing on safety-critical interactions. We introduce metrics for mode collapse, mode correctness, and coverage, emphasizing the sequential dimension of predictions. By testing four multi-agent trajectory prediction models, we demonstrate that mode collapse indeed happens. When looking at the sequential dimension, although prediction accuracy improves closer to interaction events, there are still cases where the models are unable to predict the correct interaction mode, even just before the interaction mode becomes inevitable. We hope that our framework can help researchers gain new insights and advance the development of more consistent and accurate prediction models, thus enhancing the safety of autonomous driving systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。