用多智能体系统帮导师高效完成个性化学业咨询。
AdvisingWise: Supporting Academic Advising in Higher Education Settings Through a Human-in-the-Loop Multi-Agent Framework
- 构建人机协同的多智能体框架，自动处理信息查询与回复起草。
- 专家评估显示回复准确率高，8位导师使用后信任度提升。
- 适合高校学业指导部门，尤其在师生比高的场景下使用。
学业指导对高校学生成功至关重要，但师生比例过高限制了导师及时提供支持，尤其在高峰期。大型语言模型（LLMs）的发展为改进指导流程带来机遇。我们提出AdvisingWise，一个通过权威机构资源和动态提问学生背景来生成可靠、个性化回复的多智能体系统，所有输出均经导师人工验证后才发送给学生。我们采用混合方法评估：（1）对20个样本问题的回复进行专家评估；（2）通过大模型评判信息检索策略；（3）8位导师参与用户研究以评估实用性。结果表明，AdvisingWise生成的回复准确且个性化。导师们在使用后初始对可靠性与个性化担忧逐渐消除，评价趋于积极。我们讨论了人机协同对学业指导实践的影响。
原文摘要 · Abstract (English)
Academic advising is critical to student success in higher education, yet high student-to-advisor ratios limit advisors' capacity to provide timely support, particularly during peak periods. Recent advances in Large Language Models (LLMs) present opportunities to enhance the advising process. We present AdvisingWise, a multi-agent system that automates time-consuming tasks, such as information retrieval and response drafting, while preserving human oversight. AdvisingWise leverages authoritative institutional resources and adaptively prompts students about their academic backgrounds to generate reliable, personalized responses. All system responses undergo human advisor validation before delivery to students. We evaluate AdvisingWise through a mixed-methods approach: (1) expert evaluation on responses of 20 sample queries, (2) LLM-as-a-judge evaluation of the information retrieval strategy, and (3) a user study with 8 academic advisors to assess the system's practical utility. Our evaluation shows that AdvisingWise produces accurate, personalized responses. Advisors reported increasingly positive perceptions after using AdvisingWise, as their initial concerns about reliability and personalization diminished. We conclude by discussing the implications of human-AI synergy on the practice of academic advising.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。