用检索增强生成技术打造高校多模态智能助手，解决信息难获取问题
Towards Developing a Multimodal Chat Assistant for University Stakeholders: RAG-based Approach
- 结合大模型与语义检索，从校内资源生成上下文相关回答
- 幻觉率从31.7%降至6.6%，图像输入响应稍慢但满意度高
- 支持图文混合输入，适合在低算力设备部署，适合高校管理者
高校利益相关者常面临及时可靠信息获取困难，尤其在发展中国家，智能支持系统极为稀缺。现有基于规则的聊天机器人无法处理复杂、领域特定的问题，且难以适应不断变化的机构政策。为此，我们提出一种基于检索增强生成（RAG）的多模态高校聊天机器人。该系统融合大语言模型与语义检索，从以学校为中心的资源（如大学手册）中生成上下文相关的回应。系统支持通过视觉-语言模型接收文本和图像查询，并采用量化推理实现受限硬件上的快速部署。基于FastAPI构建的可扩展后端与使用Next.js开发的响应式前端，确保了实时可用性。多模态评估显示，尽管视觉输入响应时间增加，系统在文本与图像查询上均保持较高满意度。定量评估表明，幻觉率从31.7%降低至6.6%，验证了检索增强的有效性。
原文摘要 · Abstract (English)
University stakeholders often face difficulties in accessing timely and reliable information, especially in developing countries, where there are very few intelligent support systems. Existing rule-based chatbots are unable to handle complex, domain-specific queries and are not well-equipped to adapt to evolving institutional policies. As a fill-in-the-gap solution, we present the multimodal university chatbot with retrieval-augmented generation. The system combines the large language model with semantic retrieval to produce context-based responses from institution-centric resources, such as the university handbook. The system accepts text and image queries through the vision-language model and applies quantized inference for rapid deployment on constrained hardware. A scalable backend built with FastAPI, adjoined with a responsive frontend developed with Next.js, ensures real-time usability. Our multimodal evaluation demonstrates that the system maintains strong satisfaction scores across both text and image queries, despite increased response time for visual inputs. Furthermore, quantitative evaluation shows that hallucination is reduced from 31.7% to 6.6% in our proposed RAG-based system, confirming the effectiveness of retrieval grounding.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。