用鱼眼相机实现360°多人注视方向估计，解决畸变和视角变化难题。
GazeOnce360: Fisheye-Based 360° Multi-Person Gaze Estimation with Global-Local Feature Fusion
- 采用旋转卷积与双眼关键点监督，应对鱼眼图像畸变
- 双分辨率结构融合全局上下文与局部眼球细节，提升精度
- 适合会议室、公共空间等多人群体注视分析场景
我们提出GazeOnce360，一种基于单个桌面安装向上视角鱼眼相机的端到端多人群体注视方向估计模型。不同于依赖前向摄像头且视角受限的传统方法，本工作首次探索从向上鱼眼视角估计分布于360°场景中多个人的3D注视方向。为支持该研究，我们构建了MPSGaze360——一个基于Unreal Engine生成的大规模合成数据集，包含多样化的多人群体配置，提供精确的3D注视方向与眼关键点标注。模型通过引入旋转卷积和眼关键点监督，有效缓解鱼眼图像中的严重畸变与视角变化问题。为进一步捕捉对注视估计至关重要的精细眼特征，我们设计了双分辨率架构，融合低分辨率全局上下文与高分辨率局部眼部区域特征。实验验证了各组件的有效性。本工作证明了鱼眼相机在真实多人群体场景中实现360°注视估计的可行性与潜力。
原文摘要 · Abstract (English)
We present GazeOnce360, a novel end-to-end model for multi-person gaze estimation from a single tabletop-mounted upward-facing fisheye camera. Unlike conventional approaches that rely on forward-facing cameras in constrained viewpoints, we address the underexplored setting of estimating the 3D gaze direction of multiple people distributed across a 360° scene from an upward fisheye perspective. To support research in this setting, we introduce MPSGaze360, a large-scale synthetic dataset rendered using Unreal Engine, featuring diverse multi-person configurations with accurate 3D gaze and eye landmark annotations. Our model tackles the severe distortion and perspective variation inherent in fisheye imagery by incorporating rotational convolutions and eye landmark supervision. To better capture fine-grained eye features crucial for gaze estimation, we propose a dual-resolution architecture that fuses global low-resolution context with high-resolution local eye regions. Experimental results demonstrate the effectiveness of each component in our model. This work highlights the feasibility and potential of fisheye-based 360° gaze estimation in practical multi-person scenarios. Project page: https://caizhuojiang.github.io/GazeOnce360/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。