提出图记忆框架,用可靠原型图实现跨模态可解释推理
Graph Memory: A Structured and Interpretable Framework for Modality-Agnostic Embedding-Based Inference

- 用带可靠性标注的原型图结构化嵌入空间
- 在多个数据集上准确率媲美kNN且内存小10倍
- 适合需要可解释性与多模态融合的医学分析场景
我们提出图记忆(GM),一种结构化的非参数框架,通过紧凑的、带有可靠性标注的原型区域图来表示嵌入空间。GM通过原型关系编码局部几何与区域模糊性,并通过在该结构上扩散查询证据进行推理,将实例检索、基于原型的推理与图扩散统一于一个可归纳且可解释的模型中。该框架天然具备模态无关性:在多模态场景下,为每种模态独立构建原型图,并通过可靠性感知的晚期融合整合其预测,实现对全切片图像与基因表达谱等异构数据的透明融合。在合成基准、乳腺病理(IDC)及多模态AURORA数据集上的实验表明,GM在准确率上匹配或超过kNN和标签传播,同时提供显著更优的校准性能、更平滑的决策边界,且内存占用缩小一个数量级。通过显式建模区域可靠性与关系结构,GM为单模态与多模态领域提供了原则性且可解释的非参数推理方法。
原文摘要 · Abstract (English)
We introduce Graph Memory (GM), a structured non-parametric framework that represents an embedding space through a compact graph of reliability-annotated prototype regions. GM encodes local geometry and regional ambiguity through prototype relations and performs inference by diffusing query evidence across this structure, unifying instance retrieval, prototype-based reasoning, and graph diffusion within a single inductive and interpretable model. The framework is inherently modality-agnostic: in multimodal settings, independent prototype graphs are constructed for each modality and their calibrated predictions are combined through reliability-aware late fusion, enabling transparent integration of heterogeneous sources such as whole-slide images and gene-expression profiles. Experiments on synthetic benchmarks, breast histopathology (IDC), and the multimodal AURORA dataset show that GM matches or exceeds the accuracy of kNN and Label Spreading while providing substantially better calibration, smoother decision boundaries, and an order-of-magnitude smaller memory footprint. By explicitly modeling regional reliability and relational structure, GM offers a principled and interpretable approach to non-parametric inference across single- and multi-modal domains.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。