用指纹编码实现轻量级模型合并,提升自动驾驶3D感知鲁棒性。
CodeMerge: Codebook-Guided Model Merging for Robust Test-Time Adaptation in Autonomous Driving
- 用低维特征指纹构建代码本,代替加载完整模型进行合并
- 在nuScenes-C上端到端3D检测提升14.9% NDS,KITTI迁移提升7.6% mAP
- 无需再训练即可增强在线建图、轨迹预测等下游任务性能
在动态且不可预测的测试条件下保持自动驾驶系统中稳健的3D感知能力仍是重大挑战。现有测试时自适应(TTA)方法在高方差任务如3D目标检测中常因优化不稳定和尖锐极小值而失效。尽管基于线性模式连通性(LMC)的模型合并策略通过插值微调检查点提升了稳定性,但其计算开销大,需反复访问检查点并执行多次前向传播。本文提出CodeMerge,一种轻量且可扩展的模型合并框架,通过在紧凑的潜在空间中操作规避上述限制。不加载完整模型,而是利用源模型倒数第二层特征生成每个检查点的低维指纹,并构建键值代码本。通过在这些指纹上计算岭杠杆分数来确定合并系数,实现高效模型组合且不牺牲适应质量。该方法在多个挑战性基准上表现优异,在nuScenes-C上端到端3D检测提升14.9% NDS,LiDAR检测在nuScenes-to-KITTI迁移任务中提升超过7.6% mAP,同时在无需训练的情况下改善了在线建图、运动预测和规划等下游任务性能。代码与预训练模型已发布于补充材料中。
原文摘要 · Abstract (English)
Maintaining robust 3D perception under dynamic and unpredictable test-time conditions remains a critical challenge for autonomous driving systems. Existing test-time adaptation (TTA) methods often fail in high-variance tasks like 3D object detection due to unstable optimization and sharp minima. While recent model merging strategies based on linear mode connectivity (LMC) offer improved stability by interpolating between fine-tuned checkpoints, they are computationally expensive, requiring repeated checkpoint access and multiple forward passes. In this paper, we introduce CodeMerge, a lightweight and scalable model merging framework that bypasses these limitations by operating in a compact latent space. Instead of loading full models, CodeMerge represents each checkpoint with a low-dimensional fingerprint derived from the source model's penultimate features and constructs a key-value codebook. We compute merging coefficients using ridge leverage scores on these fingerprints, enabling efficient model composition without compromising adaptation quality. Our method achieves strong performance across challenging benchmarks, improving end-to-end 3D detection 14.9% NDS on nuScenes-C and LiDAR-based detection by over 7.6% mAP on nuScenes-to-KITTI, while benefiting downstream tasks such as online mapping, motion prediction and planning even without training. Code and pretrained models are released in the supplementary material.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。