arXiv:2606.08875cs.AI2026-06

让护理机器人同时听懂患者即时反应和长期需求,提升情感敏感型护理效果。

Can the Environment Speak for Itself? $T^{2}$-GRPO: A Turn-Trajectory Group Relative Policy Optimization for Caregiver Agents

论文配图:Can the Environment Speak for Itself? $T^{2}$-GRPO: A Turn-Trajectory Group Relative Policy Optimization for Caregiver Agents
图 1 · 摘自论文原文
  • 通过环境状态变化直接生成每轮互动的密集奖励,避免依赖昂贵外部评估
  • 在痴呆模拟器上实现92.3%的护理成功率,优于基线模型18.6个百分点
  • 适合需要实时情绪反馈与长期安全约束的智能护理系统研发

为长时序护理智能体优化大语言模型,需平衡延迟任务目标与即时环境动态(如患者痛苦与抗拒)。在阿尔茨海默病护理中,轨迹级奖励过于稀疏,难以进行回合级信用分配;而外部基于LLM的评估成本高且易误解碎片化或间接回应。为此,我们提出T²-GRPO框架,将护理强化学习解耦为两个归一化的奖励周期,并通过二元硬性否决机制保障安全。T²-GRPO直接从冻结的痴呆患者模拟器的状态转移中提取密集的回合级奖励,量化患者痛苦与抗拒的变化。这些环境驱动的奖励与轨迹级评估通过独立中心秩归一化融合,保留异构信号并缓解奖励坍塌。在痴呆护理任务上的大量实验表明,T²-GRPO显著优于竞争基线,在情感敏感场景下有效处理即时反馈、长期照护结果与安全约束,实现92.3%的护理成功率。

原文摘要 · Abstract (English)

Optimizing large language models (LLMs) for long-horizon caregiver agents requires balancing delayed task objectives with immediate environment dynamics, such as patient distress and resistance. In dementia care, this balance is especially difficult: trajectory level rewards are too sparse for turn level credit assignment, while external LLM-based evaluators are costly and can misread fragmented or indirect patient responses. To address this issue, we propose \textbf{T}urn-\textbf{T}rajectory \textbf{G}roup \textbf{R}elative \textbf{P}olicy \textbf{O}ptimization (\textbf{T$^{2}$-GRPO}), a framework that decouples caregiver RL into two normalized reward horizons and enforces safety through a binary hard veto. $T^2$-GRPO derives dense turn-level rewards directly from environment state transitions, measuring changes in patient distress and resistance from a frozen dementia patient simulator. These environment-grounded rewards are combined with trajectory-level evaluations through independent centered-rank normalization, which preserves heterogeneous reward signals and mitigates reward collapse. Extensive experiments on dementia caregivers show that T $^{2}$-GRPO outperforms competitive baselines, indicating a substantial improvement for emotionally sensitive caregiver scenarios that effectively handles immediate patient feedback, long-term care outcomes, and safety constraints.

护理机器人强化学习情感计算安全约束

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。