提出动态关注机制,高效处理超大病理图像的长距离依赖
Context Matters: Query-aware Dynamic Long Sequence Modeling of Gigapixel Images
- 根据查询自适应选择关键上下文区域,仅计算相关注意力
- 在10个数据集上优于现有方法,准确率提升显著
- 适合需要高精度分析的大规模病理图像任务
全切片图像(WSI)分析因百万级图像块带来巨大计算挑战。尽管变换器通过自注意力建模长程依赖表现优异,但其二次复杂度使其在数字病理学中难以应用。现有局部-全局或线性注意力方法虽降低开销,却削弱了完整自注意力的建模能力。本文提出Querent——一种查询感知的长上下文动态建模框架,理论上可逼近全自注意力,同时实现实际效率。该方法自适应预测每个图像块最相关的周边区域,仅对潜在重要上下文进行注意力计算。结合高效的区域元数据计算与重要性估计,大幅降低计算开销,同时保留全局感知以建模细粒度块间关联。在超过10个WSI数据集上的生物标志物预测、基因突变预测、癌症分型和生存分析任务中,性能均优于当前最优方法。代码已开源:https://github.com/dddavid4real/Querent。
原文摘要 · Abstract (English)
Whole slide image (WSI) analysis presents significant computational challenges due to the massive number of patches in gigapixel images. While transformer architectures excel at modeling long-range correlations through self-attention, their quadratic computational complexity makes them impractical for computational pathology applications. Existing solutions like local-global or linear self-attention reduce computational costs but compromise the strong modeling capabilities of full self-attention. In this work, we propose Querent, i.e., the query-aware long contextual dynamic modeling framework, which achieves a theoretically bounded approximation of full self-attention while delivering practical efficiency. Our method adaptively predicts which surrounding regions are most relevant for each patch, enabling focused yet unrestricted attention computation only with potentially important contexts. By using efficient region-wise metadata computation and importance estimation, our approach dramatically reduces computational overhead while preserving global perception to model fine-grained patch correlations. Through comprehensive experiments on biomarker prediction, gene mutation prediction, cancer subtyping, and survival analysis across over 10 WSI datasets, our method demonstrates superior performance compared to the state-of-the-art approaches. Codes are available at https://github.com/dddavid4real/Querent.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。