arXiv:2603.11940cs.LG2026-03

通过全量电路追踪揭示单细胞模型的冗余与层级调控机制。

Exhaustive Circuit Mapping of a Single-Cell Foundation Model Reveals Massive Redundancy, Heavy-Tailed Hub Architecture, and Layer-Dependent Differentiation Control

  • 全层特征穷举追踪,发现连接呈重尾分布,1.8%特征占主导
  • 三阶组合消融显示冗余随交互阶数递增,无协同效应
  • 晚期层特征可强制细胞向成熟分化,证明层级控制因果性

生物基础模型的机制可解释性长期依赖于选择性特征采样、成对交互测试和观测轨迹分析,这些方法易引入系统偏差。本文在基于Transformer的单细胞基础模型Geneformer中开展三项实验:首先,在第5层对全部4065个活跃稀疏自编码器特征进行穷尽式追踪,获得1393,850条显著下游连接,较选择性采样提升27倍;结果显示连接分布呈现重尾特征,仅1.8%的特征承担了大部分连接,且40%的前20个枢纽特征缺乏生物学注释,表明先前选择性分析存在系统性注释偏差。其次,对8组特征三元组进行三阶组合消融,发现冗余随交互阶数单调增强,三阶冗余比为0.59,低于成对冗余比0.74,且无协同效应,证实模型架构在所有测试阶次下均为亚加性。第三,通过轨迹引导特征操控,建立层位置与分化方向性的因果联系:第17层特征始终推动细胞状态向成熟演化(正向比例=1.0),而第0层和第11层特征多导致远离成熟(正向比例0.00~0.58)。上述结果从相关性推进至因果证据,揭示了层依赖的细胞状态调控机制。

原文摘要 · Abstract (English)

Mechanistic interpretability of biological foundation models has relied on selective feature sampling, pairwise interaction testing, and observational trajectory analysis. Each of these can introduce systematic bias. Here we present three experiments that address these limitations through exhaustive circuit tracing, higher order combinatorial ablation, and causal trajectory steering in Geneformer, a transformer based single cell foundation model. First, exhaustive tracing of all 4065 active sparse autoencoder features at layer 5 yields 1393850 significant downstream edges, a 27 fold expansion over selective sampling. This reveals a heavy tailed hub distribution in which 1.8 percent of features account for disproportionate connectivity and 40 percent of the top 20 hubs lack biological annotation. These results indicate systematic annotation bias in prior selective analyses. Second, three way combinatorial ablation across 8 feature triplets shows that redundancy deepens monotonically with interaction order, with a three way ratio of 0.59 versus a pairwise ratio of 0.74, and with zero synergy. This confirms that the model architecture is subadditive at all tested orders. Third, trajectory guided feature steering establishes a causal link between layer position and differentiation directionality. Late layer features at L17 consistently push cell states toward maturity, with fraction positive equal to 1.0. Early and mid layer features at L0 and L11 mostly push away from maturity, with fraction positive ranging from 0.00 to 0.58. Together these results move from correlation toward causal evidence for layer dependent control of cell state.

单细胞模型可解释性冗余机制层级调控

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。