用四套指南评估AI反馈对医患沟通的改进效果
Evaluating the role of `Constitutions' for learning from AI feedback
- 用四种不同详细程度的指南指导AI评阅医患对话
- 详细指南显著提升情感表达质量，但对信息收集无效
- 适合研究AI反馈在情感类任务中的边界与局限
大语言模型（LLMs）能力提升使其可替代人类反馈用于训练和评估其他模型。这些方法常依赖于‘宪法’——由批判模型使用的书面指导原则，以提供反馈并改进生成内容。本文通过四种不同宪法，在医患沟通场景中测试其对患者中心交流的改进效果。215名人类评分者进行成对比较发现，详细宪法显著提升了情感相关表现；然而，所有宪法均未在信息获取与传递等实践性技能上超越基线模型。结果表明，尽管详细宪法应优先采用，但在某些领域，AI反馈作为奖励信号的效果存在明显局限。
原文摘要 · Abstract (English)
The growing capabilities of large language models (LLMs) have led to their use as substitutes for human feedback for training and assessing other LLMs. These methods often rely on `constitutions', written guidelines which a critic model uses to provide feedback and improve generations. We investigate how the choice of constitution affects feedback quality by using four different constitutions to improve patient-centered communication in medical interviews. In pairwise comparisons conducted by 215 human raters, we found that detailed constitutions led to better results regarding emotive qualities. However, none of the constitutions outperformed the baseline in learning more practically-oriented skills related to information gathering and provision. Our findings indicate that while detailed constitutions should be prioritised, there are possible limitations to the effectiveness of AI feedback as a reward signal in certain areas.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。