用语言和频域信息提升驾驶视线估计的准确性与鲁棒性
LISA: Language-guided Interference-aware Spatial-Frequency Attention for Driver Gaze Estimation

- 结合频域稳定特征与视觉语言知识,设计双域融合机制
- 在两个数据集上达到当前最优性能,抗遮挡和光照变化能力更强
- 适合关注驾驶安全、视觉注意力建模的研究者
驾驶视线估计是现代监控系统评估驾驶员专注度的核心指标。现有方法易受突发光照变化和传感器噪声影响,且空间模型难以区分真实视线线索与无关视觉属性。本文提出LISA框架,利用语言引导与干扰感知的空间-频率注意力机制,融合频域先验与视觉语言知识。观察到幅值谱在空间扰动下保持稳定,设计双域融合机制,将低频语义融入高频细节,并通过空间注意力精准定位眼区。为减少语义模糊,引入训练时解耦策略:使用冻结的CLIP编码器与正交正则化,显式分离视线特征与外观干扰。在两个基准数据集上的实验表明,LISA实现当前最优性能,显著提升对遮挡和光照变化的鲁棒性。代码已公开于https://github.com/Mason-bupt/LISA。
原文摘要 · Abstract (English)
Driver gaze estimation serves as a fundamental metric for evaluating driver attentiveness in modern monitoring systems. Beyond being vulnerable to sudden lighting changes and sensor noise, spatial-domain models struggle to disentangle authentic gaze cues from irrelevant visual attributes. In this paper, we propose LISA, a \textbf{L}anguage-guided \textbf{I}nterference-aware \textbf{S}patial-Frequency \textbf{A}ttention framework that combines frequency-domain priors with vision-language knowledge. Observing that the amplitude spectrum remains relatively stable even under spatial perturbations, we design a dual-domain fusion mechanism. It integrates stable low-frequency semantics into high-frequency details, employing spatial attention to precisely target ocular regions. To reduce semantic ambiguity, we also introduce a training-time disentanglement strategy. Using a frozen CLIP encoder and orthogonal regularization, we explicitly separate gaze features from appearance interference. Experiments on two benchmarks show that LISA achieves state-of-the-art performance, with significantly improved robustness against occlusions and lighting variations. The code repository is available at https://github.com/Mason-bupt/LISA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。