arXiv:2606.13267cs.CVcs.CL2026-06

手机对准文物实时识别并问答,支持中英文双语。

TimeLens: On-Device Artifact Recognition with Retrieval-Augmented Question Answering for the Grand Egyptian Museum

  • 用高质量标注数据训练轻量级模型,实现手机端实时精准识别
  • 构建108条知识库,结合检索增强生成,问答延迟降至10秒内
  • 专为博物馆场景优化,避免错误历史说法,适合游客使用

TimeLens 是一款面向大埃及博物馆(GEM)的AI双语移动端导览系统。用户将手机对准展品时,可实时识别文物并用英/阿双语提问。针对馆内部署三大挑战:51件高度相似文物(如拉美西斯雕像)难以区分、训练数据与手持拍摄条件差异大、以及模型可能输出无依据的历史信息。首先,通过数据质量驱动的迭代优化——从基础模型自动标注(YOLO-World),经空间标签清洗规则,最终形成全人工标注数据集——显著提升识别效果:最终的 YOLOv8n 模型在保持 5.97 MB 大小、可在中端手机实时运行的前提下,达到 [email protected] = 0.995,[email protected]:0.95 = 0.924。其次,采用基于 108 条记录的 ChromaDB 知识库,通过检索增强生成(RAG)架构,对比七种语言模型后选定 Gemma 4 E2B (Q4 K M),经十项优化后,端到端延迟由30秒以上降至约10秒。两模块集成于生产级 Flutter 应用,支持双语界面、馆内定位限制与语音播报。

原文摘要 · Abstract (English)

TimeLens is an AI-powered bilingual mobile guide for the Grand Egyptian Museum (GEM). Pointing a phone at an exhibit, a visitor sees the artifact recognized in real time and can ask follow-up questions answered in English or Arabic. The work addresses three problems specific to in-gallery deployment: fine-grained visual similarity among 51 catalogued artifacts (many near-identical Ramesside statues), the gap between curated training data and handheld camera conditions, and the risk of an AI guide stating unsupported historical facts. Two engineering contributions are reported. First, an on-device artifact detector was developed through a data-quality-driven iteration study -- from foundation-model auto-annotation (YOLO-World), through spatial label-cleaning rules, to a fully hand-annotated dataset -- isolating label quality as the decisive factor: the final YOLOv8n model resolves every previously failing class while remaining a 5.97 MB TensorFlow Lite asset that runs in real time on a mid-range phone ([email protected] = 0.995, [email protected]:0.95 = 0.924). Second, a bilingual Retrieval-Augmented Generation (RAG) guide, grounded in a 108-record ChromaDB knowledge base, was benchmarked across seven candidate language models, with Gemma 4 E2B (Q4 K M) selected; ten targeted optimizations reduce end-to-end latency from over 30 s to approximately 10 s. Both subsystems are integrated in a production Flutter application with bilingual interface, museum location gating, and text-to-speech support.

移动导览文物识别RAG多模态

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。