用灵活输入提升乳腺癌HER2预测准确率，单图也能用
HER2 Expression Prediction with Flexible Multi-Modal Inputs via Dynamic Bidirectional Reconstruction
- 根据输入情况自动选择补全或联合推理，支持单/双模态输入
- 仅用H&E图像时准确率达94.25%，双模态下达95.09%
- 适合资源有限地区，降低临床获取多模态图像成本
在乳腺癌HER2评估中，临床依赖H&E和IHC图像的联合分析，但两者同步获取常受临床限制与成本制约。本文提出一种自适应双模态预测框架，通过两项核心创新实现灵活输入：动态分支选择器根据输入可用性激活模态补全或联合推理；跨模态生成对抗网络（CM-GAN）可重建缺失模态的特征空间。该设计将仅使用H&E图像时的准确率从71.44%提升至94.25%，双模态输入下达95.09%，单模态条件下仍保持90.28%的可靠性。'双模态优先，单模态兼容'的架构在无需强制同步采集的前提下，实现接近双模态的性能，为资源受限地区提供低成本、高可及性的HER2评估方案。
原文摘要 · Abstract (English)
In breast cancer HER2 assessment, clinical evaluation relies on combined H&E and IHC images, yet acquiring both modalities is often hindered by clinical constraints and cost. We propose an adaptive bimodal prediction framework that flexibly supports single- or dual-modality inputs through two core innovations: a dynamic branch selector activating modality completion or joint inference based on input availability, and a cross-modal GAN (CM-GAN) enabling feature-space reconstruction of missing modalities. This design dramatically improves H&E-only accuracy from 71.44% to 94.25%, achieves 95.09% with full dual-modality inputs, and maintains 90.28% reliability under single-modality conditions. The "dual-modality preferred, single-modality compatible" architecture delivers near-dual-modality accuracy without mandatory synchronized acquisition, offering a cost-effective solution for resource-limited regions and significantly improving HER2 assessment accessibility.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。