用后验优化轨迹加速大视觉模型在街景语义理解中的训练,提升泛化能力。
Enhancing Large Vision Model in Street Scene Semantic Understanding through Leveraging Posterior Optimization Trajectory
- 通过预训练大视觉模型结合后验优化轨迹引导,提升模型对街景的语义理解能力。
- 相比现有最优方法,性能提升超66.48%,收敛速度加快6倍以上。
- 适合自动驾驶感知系统持续迭代场景,降低车载计算负担。
为提升自动驾驶感知模型的泛化能力,车辆需基于持续收集的数据动态更新模型。随着训练数据量增长,模型泛化能力显著增强,但当数据量超过模型拟合能力时,易出现欠拟合问题。为此,本文采用预训练大视觉模型(LVMs)作为主干网络,并搭配下游感知头以理解自动驾驶语义信息。该设计凭借大视觉模型强大的拟合能力与丰富多样的训练数据,有效缓解欠拟合问题并提升泛化性能。同时,为减轻车载设备在运行大视觉模型主干时训练感知头的计算负担,提出一种后验优化轨迹引导(POTGui)优化方案。具体地,引入后验优化轨迹生成器(POTGen),提前生成未来优化方向以指导当前迭代,使模型在10个训练周期内即可完成收敛。大量实验表明,该方法相较现有最先进方法性能提升超过66.48%,收敛速度提升6倍以上。
原文摘要 · Abstract (English)
To improve the generalization of the autonomous driving (AD) perception model, vehicles need to update the model over time based on the continuously collected data. As time progresses, the amount of data fitted by the AD model expands, which helps to improve the AD model generalization substantially. However, such ever-expanding data is a double-edged sword for the AD model. Specifically, as the fitted data volume grows to exceed the the AD model's fitting capacities, the AD model is prone to under-fitting. To address this issue, we propose to use a pretrained Large Vision Models (LVMs) as backbone coupled with downstream perception head to understand AD semantic information. This design can not only surmount the aforementioned under-fitting problem due to LVMs' powerful fitting capabilities, but also enhance the perception generalization thanks to LVMs' vast and diverse training data. On the other hand, to mitigate vehicles' computational burden of training the perception head while running LVM backbone, we introduce a Posterior Optimization Trajectory (POT)-Guided optimization scheme (POTGui) to accelerate the convergence. Concretely, we propose a POT Generator (POTGen) to generate posterior (future) optimization direction in advance to guide the current optimization iteration, through which the model can generally converge within 10 epochs. Extensive experiments demonstrate that the proposed method improves the performance by over 66.48\% and converges faster over 6 times, compared to the existing state-of-the-art approach.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。