用智能眼镜数据测试文本识别在真实环境下的表现
Scene Text Detection and Recognition "in light of" Challenging Environmental Conditions using Aria Glasses Egocentric Vision Cameras
- 用自建数据集测试光照、距离、分辨率对文本识别的影响
- 图像放大使字符错误率从0.65降至0.48
- 结合眼动追踪可聚焦注意力区域,提升处理效率
随着可穿戴技术发展,以第一人称视觉为核心的场景文本检测与识别(STDR)成为重要方向。本文基于Meta的Project Aria智能眼镜,研究光照、距离和分辨率等环境因素对先进STDR算法在真实场景中的影响。构建了受控条件下采集的新型数据集,评估了两种OCR流程:EAST+CRNN 和 EAST+PyTesseract。结果表明,分辨率和距离显著影响识别准确率,而光照影响较难预测。值得注意的是,图像上采样作为预处理手段,将字符错误率(CER)从0.65降至0.48。此外,通过整合眼动追踪信息,可聚焦用户关注区域,优化处理效率。本研究不仅在真实条件下基准化了STDR性能,也为自适应、用户感知的增强现实系统提供了基础。研究目标是推动辅助性与科研应用中鲁棒、情境敏感的文本识别发展,如资产巡检与营养分析。代码已开源:https://github.com/josepDe/Project_Aria_STR。
原文摘要 · Abstract (English)
In an era where wearable technology is reshaping applications, Scene Text Detection and Recognition (STDR) becomes a straightforward choice through the lens of egocentric vision. Leveraging Meta's Project Aria smart glasses, this paper investigates how environmental variables, such as lighting, distance, and resolution, affect the performance of state-of-the-art STDR algorithms in real-world scenarios. We introduce a novel, custom-built dataset captured under controlled conditions and evaluate two OCR pipelines: EAST with CRNN, and EAST with PyTesseract. Our findings reveal that resolution and distance significantly influence recognition accuracy, while lighting plays a less predictable role. Notably, image upscaling emerged as a key pre-processing technique, reducing Character Error Rate (CER) from 0.65 to 0.48. We further demonstrate the potential of integrating eye-gaze tracking to optimise processing efficiency by focusing on user attention zones. This work not only benchmarks STDR performance under realistic conditions but also lays the groundwork for adaptive, user-aware AR systems. Our contributions aim to inspire future research in robust, context-sensitive text recognition for assistive and research-oriented applications, such as asset inspection and nutrition analysis. The code is available at https://github.com/josepDe/Project_Aria_STR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。