arXiv:2511.12708cs.CV2025-11被引 1

用90个样本实现驾驶员注意力预测与原因解释。

FSDAM: Few-Shot Driving Attention Modeling via Vision-Language Coupling

  • 分解注意力为场景、当前焦点、未来焦点和因果解释四部分。
  • 仅用90个标注样本,仍能准确预测注视点并生成合理解释。
  • 适合需要可解释性的人机协作自动驾驶研究。

理解驾驶员不仅看哪里,还为何转移注意力,对自动驾驶中的人机可解释协作至关重要。驾驶员注意力并非纯感知,而是具有语义结构。因此,可通过极少的语义标注而非密集大规模标注来学习注意力变化。本文提出FSDAM(Few-Shot Driver Attention Modeling),仅使用90个标注样本,即可联合完成空间注意力预测与结构化解释生成。核心思路是将注意力分解为显式推理表示:场景上下文、当前关注点、预期下一关注点及因果解释,并通过极小配对监督学习下一关注点预测。为解决现有模型的任务冲突与大样本需求问题,在少样本条件下减少任务干扰,我们设计了双路径架构,分别处理空间预测与文本生成。此外,引入仅训练阶段使用的视觉-语言对齐机制,注入语义先验而不增加推理开销,有效缓解少样本训练下的任务干扰。尽管数据极度稀缺,FSDAM在注视点预测上表现优异,并生成连贯、情境相关的结构化推理,提升可解释性。模型还在多个驾驶基准上展现出强零样本泛化能力。

原文摘要 · Abstract (English)

Understanding not only where drivers look but also why their attention shifts is essential for interpretable human-AI collaboration in autonomous driving. Driver attention is not purely perceptual but semantically structured. Thus, attention shifts can be learned through minimal semantic supervision rather than dense large-scale annotation. We present \textbf{FSDAM} (\textbf{F}ew-\textbf{S}hot \textbf{D}river \textbf{A}ttention \textbf{M}odeling), a framework that achieves joint spatial attention prediction and structured explanation generation using 90 annotated examples. Our key insight is to decompose attention into an explicit reasoning representation, including scene context, current focus, anticipated next focus, and causal explanation, and to learn next-focus anticipation through minimal-pair supervision. To address task conflict and large sample requirements of existing models, and to mitigate task interference under limited data, we introduce a novel dual-pathway architecture in which separate modules handle spatial prediction and caption generation. In addition, we use a training-only vision-language alignment mechanism that injects semantic priors into spatial learning without increasing inference complexity, mitigating task interference under few-shot training. Despite extreme data scarcity, FSDAM achieves competitive performance in gaze prediction, and generates coherent, context-aware structural reasoning for improved interpretability. The model further demonstrates strong zero-shot generalization across multiple driving benchmarks.

注意力建模少样本学习可解释性视觉语言对齐

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。