用专家反馈提升医疗图像修复,生成更真实的息肉图像。
PrefPaint: Enhancing Medical Image Inpainting through Expert Human Feedback
- 引入专家反馈优化Stable Diffusion的图像修复过程。
- 相比传统方法,显著减少视觉不一致,提升解剖准确性。
- 适合临床医生参与模型迭代,尤其适用于资源有限的医院。
图像修复在医学影像中有广泛应用,但在胃肠镜领域,生成解剖准确的合成息肉图像仍属未充分探索的问题。错误的生成结果可能导致误诊。为保障可靠性,需引入肿瘤科专家等领域专家的直接反馈。我们提出PrefPaint,一种将专家反馈融入Stable Diffusion图像修复的交互系统。采用D3PO而非完整强化学习人类反馈(RLHF),避免昂贵的奖励模型训练,更适合计算资源受限的临床环境。同时开发了轻量级网页界面,核心是模型树版本管理界面,可视化微调模型的演化过程,提升专家反馈与模型管理的直观性。用户研究表明,PrefPaint在减少视觉不一致方面优于现有方法,生成高度真实、解剖准确的息肉图像,适用于临床AI应用。
原文摘要 · Abstract (English)
Inpainting, the process of filling missing or corrupted image parts, has broad applications in medical imaging. However, generating anatomically accurate synthetic polyp images for clinical AI is a largely underexplored problem. In specialized fields like gastroenterology, inaccuracies in generated images can lead to false patterns and significant errors in downstream diagnosis. To ensure reliability, models require direct feedback from domain experts like oncologists. We propose PrefPaint, an interactive system that incorporates expert human feedback into Stable Diffusion Inpainting. By using D3PO instead of full RLHF, our approach bypasses the need for computationally expensive reward models, making it a highly practical choice for resource-constrained clinical settings. Furthermore, we introduce a streamlined web-based interface to facilitate this expert-in-the-loop training. Central to this platform is the Model Tree versioning interface, a novel HCI concept that visualizes the evolutionary progression of fine-tuned models. This interactive interface provides a smooth and intuitive user experience, making it easier to offer feedback and manage the fine-tuning process. User studies show that PrefPaint outperforms existing methods, reducing visual inconsistencies and generating highly realistic, anatomically accurate polyp images suitable for clinical AI applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。