EyeSeg通过建模不确定性提升AR/VR中眼部分割精度,尤其在模糊和遮挡下表现更优。
EyeSeg: An Uncertainty-Aware Eye Segmentation Framework for AR/VR
- 引入贝叶斯不确定性学习,显式建模分割置信度
- 在运动模糊、眼皮遮挡等场景下,MIoU等指标显著优于旧方法
- 输出不确定度分数,适用于对鲁棒性要求高的交互系统
基于增强现实(AR)与虚拟现实(VR)的人机交互日益普及,其流畅体验依赖于高精度的眼动估计,而眼动估计的准确性又高度依赖于眼部分割的精确性。本文提出EyeSeg,一种面向AR/VR的不确定性感知眼部分割框架,旨在解决现有方法在运动模糊、眼皮遮挡及训练测试域差异下的性能瓶颈。我们观察到这些挑战可被不确定性量化,因此设计了基于封闭集先验的贝叶斯后验不确定性学习机制。理论上证明了学习后验的统计量可反映分割不确定性水平,并在下游任务(如眼动估计)中实证优于现有方法。EyeSeg同时输出分割结果与不确定性分数,通过加权融合多组眼动估计以提升鲁棒性,在运动模糊、眼皮遮挡及跨域场景下均表现优异。实验表明,该方法在MIoU、E1、F1和ACC等指标上全面超越此前方案。代码已开源:https://github.com/JethroPeng/EyeSeg。
原文摘要 · Abstract (English)
Human-machine interaction through augmented reality (AR) and virtual reality (VR) is increasingly prevalent, requiring accurate and efficient gaze estimation which hinges on the accuracy of eye segmentation to enable smooth user experiences. We introduce EyeSeg, a novel eye segmentation framework designed to overcome key challenges that existing approaches struggle with: motion blur, eyelid occlusion, and train-test domain gaps. In these situations, existing models struggle to extract robust features, leading to suboptimal performance. Noting that these challenges can be generally quantified by uncertainty, we design EyeSeg as an uncertainty-aware eye segmentation framework for AR/VR wherein we explicitly model the uncertainties by performing Bayesian uncertainty learning of a posterior under the closed set prior. Theoretically, we prove that a statistic of the learned posterior indicates segmentation uncertainty levels and empirically outperforms existing methods in downstream tasks, such as gaze estimation. EyeSeg outputs an uncertainty score and the segmentation result, weighting and fusing multiple gaze estimates for robustness, which proves to be effective especially under motion blur, eyelid occlusion and cross-domain challenges. Moreover, empirical results suggest that EyeSeg achieves segmentation improvements of MIoU, E1, F1, and ACC surpassing previous approaches. The code is publicly available at https://github.com/JethroPeng/EyeSeg.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。