用大模型+触觉反馈帮视障者实时导航，识别准确率超80%。
LLM-Glasses: GenAI-driven Glasses with Haptic Feedback for Navigation of Visually Impaired People
- 结合目标检测与大模型推理，将视觉信息转为颞部触觉提示。
- 在无障碍物场景下决策准确率达91.8%，动态障碍下仍达81.5%。
- 适合视障辅助设备研究者，推动智能可穿戴落地应用。
LLM-Glasses 是一种可穿戴导航系统，通过 YOLO-World 实现物体检测，利用 GPT-4o 进行语义推理，并结合触觉反馈实现实时引导。该设备将视觉场景理解转化为颞部的直观触感提示，支持双手自由操作。三项实验评估了系统性能：13 种触觉模式识别平均准确率为 81.3%；基于 VICON 的路径引导实验中，使用触觉提示成功完成预设路线；在大模型驱动的场景评估中，无障碍物时决策准确率达 91.8%，静态障碍下为 84.6%，动态障碍下为 81.5%。结果表明，该系统在受控环境中可提供可靠导航支持，未来需提升响应速度并拓展至更复杂的真实场景。
原文摘要 · Abstract (English)
LLM-Glasses is a wearable navigation system which assists visually impaired people by utilizing YOLO-World object detection, GPT-4o-based reasoning, and haptic feedback for real-time guidance. The device translates visual scene understanding into intuitive tactile feedback on the temples, allowing hands-free navigation. Three studies evaluate the system: recognition of 13 haptic patterns with an average recognition rate of 81.3%, VICON-based guidance with predefined paths using haptic cues, and an LLM-guided scene evaluation with decision accuracies of 91.8% without obstacles, 84.6% with static obstacles, and 81.5% with dynamic obstacles. These results show that LLM-Glasses can deliver reliable navigation support in controlled environments and motivate further work on responsiveness and deployment in more complex real-world scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。