PanFoMa用轻量混合模型提升癌症单细胞分析效率与精度。
PanFoMa: A Lightweight Foundation Model and Benchmark for Pan-Cancer
- 结合Transformer与状态空间模型,兼顾局部基因交互与全局序列建模。
- 在33种癌症、超350万细胞上表现优于现有模型,关键任务提升超7%。
- 配套大型基准数据集PanFoMaBench,适合癌症研究与多组学分析者使用。
单细胞RNA测序对解析肿瘤异质性至关重要,但泛癌研究仍面临两大挑战:学习判别性强且高效的单细胞表示,以及建立全面的评估基准。本文提出PanFoMa,一种轻量级混合神经网络,融合Transformer与状态空间模型优势,在性能与效率间取得平衡。PanFoMa包含前端共享自注意力层以捕捉复杂、无序的基因互作;后端线性时间状态空间模型高效整合全局上下文,实现转录组建模,有效捕获局部与全局调控信号。为支持可靠评估,我们构建了大规模泛癌单细胞基准PanFoMaBench,涵盖33种癌症亚型,超过350万高质量细胞,经严格预处理流程筛选。实验表明,PanFoMa在本泛癌基准上优于现有模型(+4.0%),并在多个公开任务中表现优异:细胞类型注释(+7.4%)、批次整合(+4.0%)和多组学整合(+3.1%)。代码已开源。
原文摘要 · Abstract (English)
Single-cell RNA sequencing (scRNA-seq) is essential for decoding tumor heterogeneity. However, pan-cancer research still faces two key challenges: learning discriminative and efficient single-cell representations, and establishing a comprehensive evaluation benchmark. In this paper, we introduce PanFoMa, a lightweight hybrid neural network that combines the strengths of Transformers and state-space models to achieve a balance between performance and efficiency. PanFoMa consists of a front-end local-context encoder with shared self-attention layers to capture complex, order-independent gene interactions; and a back-end global sequential feature decoder that efficiently integrates global context using a linear-time state-space model. This modular design preserves the expressive power of Transformers while leveraging the scalability of Mamba to enable transcriptome modeling, effectively capturing both local and global regulatory signals. To enable robust evaluation, we also construct a large-scale pan-cancer single-cell benchmark, PanFoMaBench, containing over 3.5 million high-quality cells across 33 cancer subtypes, curated through a rigorous preprocessing pipeline. Experimental results show that PanFoMa outperforms state-of-the-art models on our pan-cancer benchmark (+4.0\%) and across multiple public tasks, including cell type annotation (+7.4\%), batch integration (+4.0\%) and multi-omics integration (+3.1\%). The code is available at https://github.com/Xiaoshui-Huang/PanFoMa.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。