用眼动数据自动标注鼠类行为视频，准确率提升70%
Eye on the Target: Eye Tracking Meets Rodent Tracking
- 用Aria眼镜眼动数据生成分割提示点
- 结合后处理使分割准确率提升至66.2（Jaccard）
- 适合需要高效动物行为分析的研究者
从视频记录中分析动物行为对科学研究至关重要，但人工标注仍耗时且易受主观影响。本文提出一种新流程，利用Aria眼镜的眼动数据生成提示点，并通过快速零样本分割模型生成分割掩码。同时应用后处理优化提示，显著提升分割质量。实验表明，该方法在大鼠数据集上将分割结果的交并比（Jaccard Index）从38.8提升至66.2，改进幅度达70.6%。
原文摘要 · Abstract (English)
Analyzing animal behavior from video recordings is crucial for scientific research, yet manual annotation remains labor-intensive and prone to subjectivity. Efficient segmentation methods are needed to automate this process while maintaining high accuracy. In this work, we propose a novel pipeline that utilizes eye-tracking data from Aria glasses to generate prompt points, which are then used to produce segmentation masks via a fast zero-shot segmentation model. Additionally, we apply post-processing to refine the prompts, leading to improved segmentation quality. Through our approach, we demonstrate that combining eye-tracking-based annotation with smart prompt refinement can enhance segmentation accuracy, achieving an improvement of 70.6% from 38.8 to 66.2 in the Jaccard Index for segmentation results in the rats dataset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。