arXiv:2504.02894cs.CLcs.AI2025-04被引 3

用在线强化学习让心理对话系统实时个性化,更懂用户需求。

OnRL-RAG: Real-Time Personalized Mental Health Dialogue System

  • 结合检索增强与在线强化学习,动态优化对话响应
  • 在2838名大学生数据上显著优于标准RAG和LLM
  • 适合心理健康、人机交互等需实时个性化的场景

大语言模型(LLMs)虽广泛应用,但受限于预训练数据,知识可能过时。为增强其能力,提出检索增强生成(RAG)以引入最新信息。然而,传统RAG难以实现个性化。本研究提出基于在线强化学习的检索增强生成系统(OnRL-RAG),通过反馈机制动态适应用户需求,尤其适用于心理健康的复杂多变情境。我们使用来自2028名大学生的公开数据集,每名学生回答28个调查问题,验证系统性能。实验对比GPT-4o、GPT-4o-mini、Gemini-1.5及GPT-3.5,OnRL-RAG在个性化响应上表现更优。该系统推动了大模型在日常生活中的个性化服务应用,也为社会学、心理学与神经科学提供贴近真实环境的研究支持。

原文摘要 · Abstract (English)

Large language models (LLMs) have been widely used for various tasks and applications. However, LLMs and fine-tuning are limited to the pre-trained data. For example, ChatGPT's world knowledge until 2021 can be outdated or inaccurate. To enhance the capabilities of LLMs, Retrieval-Augmented Generation (RAG), is proposed to augment LLMs with additional, new, latest details and information to LLMs. While RAG offers the correct information, it may not best present it, especially to different population groups with personalizations. Reinforcement Learning from Human Feedback (RLHF) adapts to user needs by aligning model responses with human preference through feedback loops. In real-life applications, such as mental health problems, a dynamic and feedback-based model would continuously adapt to new information and offer personalized assistance due to complex factors fluctuating in a daily environment. Thus, we propose an Online Reinforcement Learning-based Retrieval-Augmented Generation (OnRL-RAG) system to detect and personalize the responding systems to mental health problems, such as stress, anxiety, and depression. We use an open-source dataset collected from 2028 College Students with 28 survey questions for each student to demonstrate the performance of our proposed system with the existing systems. Our system achieves superior performance compared to standard RAG and simple LLM via GPT-4o, GPT-4o-mini, Gemini-1.5, and GPT-3.5. This work would open up the possibilities of real-life applications of LLMs for personalized services in the everyday environment. The results will also help researchers in the fields of sociology, psychology, and neuroscience to align their theories more closely with the actual human daily environment.

心理健康个性化强化学习对话系统

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。