用无人机监控数据训练自动驾驶,零样本跨城部署效果显著提升
SkyDrive: Learning to Drive in a New City from Aerial Traffic Monitoring

- 通过无人机俯视视角采集多车行为数据,实现高效监督信号扩展
- 137小时航拍数据生成65万条驾驶样本,跨城零样本性能下降超40%
- 仅需每地30分钟空中监测,即可显著改善模型在新城市的泛化能力
自动驾驶在大量人类驾驶示范数据上通过模仿学习取得显著进展。然而,当将训练好的规划器直接应用于新环境时,常因交通规则、道路布局和驾驶行为的域偏移导致性能严重下降。传统适应新城市需耗时耗力的本地车辆传感器数据采集。本文提出SkyDrive框架,利用无人机交通监控提供高效监督信号。相比车辆自身感知,无人机可广角同步观测多辆道路使用者,使每辆车都成为具象化驾驶行为的数据源,大幅扩展监督规模。基于137小时航拍视频,我们提取65万条驾驶样本,构建轨迹规划器与运动预测基准。零样本实验显示跨城性能差距显著(下降超40%),但仅需每地点30分钟的空中监控数据,多数模型性能即获明显改善。结果表明,航拍交通监控是适应新城市自动驾驶系统的高效且可扩展数据源。数据与代码将公开。
原文摘要 · Abstract (English)
Autonomous driving has made remarkable progress through imitation learning with massive human demonstration data. However, a trained planner often degrades severely when applied to a new environment zero-shot, because of domain shifts in traffic regulations, road layout and driving behaviors. Therefore, adapting a trajectory planner to a new city typically requires resource-demanding local data collection with a vehicle sensor suite. In this work, we show that driving behavior can be learned from a scalable and efficient alternative. We introduce \emph{SkyDrive}, a framework that utilizes drone-based traffic monitoring to provide efficient supervision for autonomous driving agents in a new environment. While vehicle-based data collection logs the ego and its surroundings, an aerial platform naturally observes many road users simultaneously over an extended field of view. As a result, every vehicle can be a data source with grounded driving behavior, effectively scaling up the amount of supervision. Based on 137 hours of aerial traffic monitoring footage, we extract 650K driving samples and construct a benchmark for trajectory planners and motion predictors. Zero-shot experiments with multiple models reveal significant cross-city domain gaps, but many of them can be alleviated by limited supervision from the sky, e.g., 30 minutes of monitoring per location. Our findings show that aerial traffic monitoring is an efficient and scalable data source for adapting autonomous driving systems in new cities. Data and code will be made publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。