用闭环微调让交通模型更真实，小模型也能超越大模型。
Closed-Loop Supervised Fine-Tuning of Tokenized Traffic Models
- 通过闭环采样选择最优轨迹，缓解真实场景中的分布偏移。
- 700万参数模型超越1.02亿参数同类模型，登顶Waymo挑战赛。
- 仅需历史数据即可训练，无需强化学习或对抗生成技术。
交通模拟旨在学习一个交通主体策略，使其在闭环运行时能准确复现真实世界中观测到的轨迹联合分布。受大语言模型启发，基于分词的多智能体策略已成为当前交通模拟的最新技术。然而，这些模型通常通过开环行为克隆训练，在仿真中闭环执行时面临协变量偏移问题。本文提出了一种名为「前K个中最接近」（CAT-K）的闭环微调策略，简单而有效。CAT-K仅需现有轨迹数据，无需强化学习或生成对抗模仿。具体而言，该方法使一个仅700万参数的分词交通模拟策略超越了同模型家族中1.02亿参数的模型，在提交时位列Waymo Sim Agent Challenge排行榜首位。代码已开源于https://github.com/NVlabs/catk。
原文摘要 · Abstract (English)
Traffic simulation aims to learn a policy for traffic agents that, when unrolled in closed-loop, faithfully recovers the joint distribution of trajectories observed in the real world. Inspired by large language models, tokenized multi-agent policies have recently become the state-of-the-art in traffic simulation. However, they are typically trained through open-loop behavior cloning, and thus suffer from covariate shift when executed in closed-loop during simulation. In this work, we present Closest Among Top-K (CAT-K) rollouts, a simple yet effective closed-loop fine-tuning strategy to mitigate covariate shift. CAT-K fine-tuning only requires existing trajectory data, without reinforcement learning or generative adversarial imitation. Concretely, CAT-K fine-tuning enables a small 7M-parameter tokenized traffic simulation policy to outperform a 102M-parameter model from the same model family, achieving the top spot on the Waymo Sim Agent Challenge leaderboard at the time of submission. The code is available at https://github.com/NVlabs/catk.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。