用地理先验指导视觉模型，提升罕见故障识别能力
GAFT: Geo-Anchored Fine-Tuning for Hazard Identification from Rare Failures
- 基于几何先验引导注意力图，高效微调视觉基础模型
- 在森林障碍物数据集上，F2分数从0.0607提升至0.3757（显著提升）
- 适合自动驾驶中稀有危险场景的鲁棒识别，尤其关注小样本学习
非铺装路面导航可能因物理结构导致无法恢复的状态（如高中心卡死或陷入），需人工干预。识别这些结构至关重要但极具挑战：失败事件稀少且采集成本高，训练数据有限；现有数据仅关联帧与结果，未标注导致失败的视觉线索。直接学习此类数据易捕捉场景特异性特征，泛化能力差。本文提出【地理锚定微调（GAFT）】，一种参数高效方法，通过几何先验引导视觉基础模型的微调。其利用空间注意力回溯图与几何先验对齐，指导LoRA适配，同时保留预训练表征。在经人工干预验证的森林障碍物基准测试中，十次独立训练的适应模型均显著优于冻结的DINOv2及监督式参数高效微调基线，重复留一场景外平均F2从0.0607提升至0.3757（配对检验显著）。其中最优的GAFT模型达到0.570的重复留一场景外F2。代码与数据集：https://github.com/Xu-Yanran/geo_anchored_fine_tuning
原文摘要 · Abstract (English)
Off-road navigation can fail when physical structures induce irrecoverable states such as high-centering or entrapment, requiring human interventions. Identifying these structures is crucial, yet challenging. Such failure events are rare and costly to collect, resulting in limited training data. Moreover, the collected data associate frames with outcomes, but do not indicate the visual cues responsible for the failure. Learning directly from these data can therefore exploit scenario-specific visual cues, leading to poor generalization. We propose \textbf{Geo-Anchored Fine-Tuning (GAFT)}, a parameter-efficient method that adapts a vision foundation model with a geometry-derived prior. It guides LoRA adaptation by aligning a spatial attention-rollout map with the geometry prior, while preserving pretrained representations. On an intervention-verified forest hazard benchmark, across ten independently trained adaptations, GAFT consistently outperforms frozen DINOv2 and supervised PEFT baselines, improving the repeated leave-one-scenario-out mean $F_2$ from 0.0607 to 0.3757 with statistical significance under paired analysis. Within these independently trained models, the best-performing GAFT model achieves a repeated-LOSO $F_2$ of 0.570. Code and benchmark: https://github.com/Xu-Yanran/geo_anchored_fine_tuning
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。