MR.NAVI用混合现实帮视障者实时听懂环境、避开障碍、坐公交。
MR.NAVI: Mixed-Reality Navigation Assistant for the Visually Impaired
- 用视觉+语音融合技术，实时解析环境并生成语音提示。
- 在陌生环境中测试显示导航和描述功能有效可用。
- 适合需要独立出行的视障人群，尤其方便乘公交出行。
全球超过4300万严重视觉障碍者在陌生环境中面临显著导航挑战。我们提出MR.NAVI，一种混合现实系统，通过实时场景理解与直观音频反馈增强视障用户的空间感知能力。系统结合计算机视觉算法进行物体检测与深度估计，利用自然语言处理提供场景上下文描述、主动避障及导航指引。分布式架构采用MobileNet进行物体检测，基于RANSAC的地板检测与DBSCAN聚类实现障碍物规避，并集成公共交通API提供公交导航信息。通过用户实验评估了在陌生环境中的场景描述与导航功能，结果表明系统具备良好的可用性与有效性。
原文摘要 · Abstract (English)
Over 43 million people worldwide live with severe visual impairment, facing significant challenges in navigating unfamiliar environments. We present MR.NAVI, a mixed reality system that enhances spatial awareness for visually impaired users through real-time scene understanding and intuitive audio feedback. Our system combines computer vision algorithms for object detection and depth estimation with natural language processing to provide contextual scene descriptions, proactive collision avoidance, and navigation instructions. The distributed architecture processes sensor data through MobileNet for object detection and employs RANSAC-based floor detection with DBSCAN clustering for obstacle avoidance. Integration with public transit APIs enables navigation with public transportation directions. Through our experiments with user studies, we evaluated both scene description and navigation functionalities in unfamiliar environments, showing promising usability and effectiveness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。