arXiv:2412.08901cs.LGcs.AI2024-12AAAI被引 29

让一份报告同时满足不同医生的偏好,提升生成结果的实用性和多样性。

Radiology Report Generation via Multi-objective Preference Optimization

  • 用多目标强化学习建模医生多元偏好,通过偏好向量控制生成风格。
  • 在两个公开数据集上达到当前最优,单一模型可适配多种临床需求。
  • 无需微调即可生成符合特定偏好报告,适合医疗场景快速部署。

自动放射科报告生成(RRG)对缓解放射科医生的工作负担至关重要。现有方法依赖监督回归或知识注入,但生成报告难以最优匹配医生偏好。由于医生偏好具有异质性与多维性——如有人更关注语句流畅性,有人更强调临床准确性——本文提出基于多目标偏好优化(MPO)的新方法,通过多维奖励函数和多目标强化学习,将预训练的RRG模型对齐至多重人类偏好。具体地,使用偏好向量表示各偏好的权重,并作为条件输入模型;通过偏好向量与多维奖励的点积得到加权奖励;再利用强化学习优化该奖励,使模型在训练阶段随机采样多样偏好向量,在整个偏好空间中学习最优策略。推理时,模型可直接生成符合特定偏好的报告,无需额外微调。在两个公开数据集上的实验表明,该方法能以单一模型适应不同偏好,且性能达当前最佳。

原文摘要 · Abstract (English)

Automatic Radiology Report Generation (RRG) is an important topic for alleviating the substantial workload of radiologists. Existing RRG approaches rely on supervised regression based on different architectures or additional knowledge injection,while the generated report may not align optimally with radiologists' preferences. Especially, since the preferences of radiologists are inherently heterogeneous and multidimensional, e.g., some may prioritize report fluency, while others emphasize clinical accuracy. To address this problem,we propose a new RRG method via Multi-objective Preference Optimization (MPO) to align the pre-trained RRG model with multiple human preferences, which can be formulated by multi-dimensional reward functions and optimized by multi-objective reinforcement learning (RL). Specifically, we use a preference vector to represent the weight of preferences and use it as a condition for the RRG model. Then, a linearly weighed reward is obtained via a dot product between the preference vector and multi-dimensional reward. Next,the RRG model is optimized to align with the preference vector by optimizing such a reward via RL. In the training stage,we randomly sample diverse preference vectors from the preference space and align the model by optimizing the weighted multi-objective rewards, which leads to an optimal policy on the entire preference space. When inference,our model can generate reports aligned with specific preferences without further fine-tuning. Extensive experiments on two public datasets show the proposed method can generate reports that cater to different preferences in a single model and achieve state-of-the-art performance.

医学报告生成多目标优化偏好学习强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。