为米兰比可卡大学学生打造智能问答机器人，用检索增强生成技术提升服务体验。
Unimib Assistant: designing a student-friendly RAG-based chatbot for all their needs
- 基于RAG架构与自定义GPT，结合校方资料和学生访谈构建专用问答系统。
- 用户认可其友好交互与结构化回答，但存在信息不准确、遗漏关键内容问题。
- 适合高校AI助手研发团队参考，尤其关注真实场景下的可用性优化。
大型语言模型（LLMs）在自然语言处理方面的能力空前强大，已广泛应用于各类任务。本试点研究通过OpenAI的自定义GPT功能，利用检索增强生成（RAG）系统对ChatGPT行为进行专业化改造。我们开发的聊天机器人名为Unimib Assistant，旨在通过问答方式满足米兰比可卡大学（Unimib）学生的具体需求。系统通过提示词明确其角色，并整合了来自六名学生访谈的初步需求调研所得的大学相关文档与链接。经过初步定制后，我们邀请另外六名学生进行了定性可用性测试，以识别系统的优缺点，为后续重设计提供依据。尽管用户普遍认为该机器人具备友好的使用体验、较高的可靠性、结构清晰的回答及自然的对话语气，但仍暴露出若干重大技术和功能缺陷。例如，系统无法始终提供完全准确的信息，常忽略上传材料中已存在的相关内容，且有时生成不可点击的链接，影响了其可信度——因为溯源信息对用户至关重要。未来将开展更深入的研究、收集更多用户反馈并进行迭代优化。
原文摘要 · Abstract (English)
Natural language processing skills of Large Language Models (LLMs) are unprecedented, having wide diffusion and application in different tasks. This pilot study focuses on specializing ChatGPT behavior through a Retrieval-Augmented Generation (RAG) system using the OpenAI custom GPTs feature. The purpose of our chatbot, called Unimib Assistant, is to provide information and solutions to the specific needs of University of Milano-Bicocca (Unimib) students through a question-answering approach. We provided the system with a prompt highlighting its specific purpose and behavior, as well as university-related documents and links obtained from an initial need-finding phase, interviewing six students. After a preliminary customization phase, a qualitative usability test was conducted with six other students to identify the strengths and weaknesses of the chatbot, with the goal of improving it in a subsequent redesign phase. While the chatbot was appreciated for its user-friendly experience, perceived general reliability, well-structured responses, and conversational tone, several significant technical and functional limitations emerged. In particular, the satisfaction and overall experience of the users was impaired by the system's inability to always provide fully accurate information. Moreover, it would often neglect to report relevant information even if present in the materials uploaded and prompt given. Furthermore, it sometimes generated unclickable links, undermining its trustworthiness, since providing the source of information was an important aspect for our users. Further in-depth studies and feedback from other users as well as implementation iterations are planned to refine our Unimib Assistant.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。