用多模态融合提升口腔病变分类准确率,尤其适合数据少的医疗场景。
Uncertainty-Aware Multimodal Fusion for Oral Lesion Classification
- 通过重建RGB图像为高光谱数据,融合影像与人口统计信息
- 在未见数据上达到66.23%宏F1和64.56%准确率
- 适合资源有限但需高鲁棒性诊断的临床筛查场景
由于标注数据稀缺,低资源环境下早期发现口腔癌及潜在恶性疾病面临重大挑战。本文提出一种统一的口腔病变分类方法,结合深度学习、光谱分析与人口统计学数据。从公开数据集中筛选出经病理学家验证的口腔腔体图像子集,使用微调的ConvNeXt-v2网络提取深层特征后,通过重建算法将其转换至高光谱域。从重建的高光谱立方体中提取血红蛋白敏感、纹理及光谱特征,并与人口统计学数据融合。采用患者级验证评估多种机器学习模型,最终设计了一种增量启发式元学习器(IHML),通过概率特征堆叠与不确定性感知的多模态表示抽象,结合患者层面平滑,实现校准基分类器的融合。通过解耦证据提取与决策融合,IHML在异质性、小样本医疗数据集上稳定预测。在未见测试集上,模型取得66.23%的宏F1和64.56%的整体准确率。结果表明,从RGB到高光谱的重建以及集成元学习能显著提升真实世界口腔病变筛查的诊断鲁棒性。
原文摘要 · Abstract (English)
Early detection of oral cancer and potentially malignant diseases is a major challenge in low-resource settings due to the scarcity of annotated data. We provide a unified approach for oral lesion classification that incorporates deep learning, spectral analysis, and demographic data. A pathologist verified subset of oral cavity images was curated from a publicly available dataset. Oral cavity pictures were processed using a fine tuned ConvNeXtv2 network for deep embeddings before being translated into the hyperspectral domain using a reconstruction algorithm. Haemoglobin sensitive, textural, and spectral descriptors were obtained from the reconstructed hyperspectral cubes and combined with demographic data. Multiple machine learning models were evaluated using patient specific validation. Finally, an incremental heuristic meta learner (IHML) was developed that merged calibrated base classifiers via probabilistic feature stacking and uncertainty-aware abstraction of multimodal representations with patient level smoothing. By decoupling evidence extraction from decision fusion, IHML stabilizes predictions in heterogeneous, small sample medical datasets. On an unseen test set, our proposed model achieved a macro F1 of 66.23% and an overall accuracy of 64.56%. The findings demonstrate that RGB to hyperspectral reconstruction and ensemble meta learning improve diagnostic robustness in real world oral lesion screening.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。