arXiv:2607.19618q-bio.GNcs.AI2026-07

用因果方法验证基因组模型中的转录因子结合特征,避免误判假阳性。

Causal dictionary learning reveals and validates transcription-factor binding features in genomic language models

论文配图:Causal dictionary learning reveals and validates transcription-factor binding features in genomic language models
图 1 · 摘自论文原文
  • 结合稀疏字典学习与因果干预,提取可解释的基因序列特征。
  • 在多个模型中发现7至14个可重复的因果性结合特征,排除了序列组成干扰。
  • 适用于想验证深度学习模型真实生物意义的研究者,无需额外实验。

基因组语言模型在调控基因组任务中表现优异,但其内部表征仍不透明,且缺乏验证模型中所谓‘概念’是否真实存在的系统方法。本文提出一种结合稀疏字典学习与因果干预的框架,用于提取、验证并因果测试基因组基础模型中的可解释特征。在两种不同架构的模型——核苷酸变换器(6-mer分词)和DNABERT-2(字节对编码)——的隐藏激活上训练top-k稀疏自编码器,成功恢复出数千个单一语义的特征,对应转录因子(TF)序列基序。我们发现,仅以位置权重矩阵进行朴素验证会严重受GC含量和重复元件干扰,导致数百个虚假‘TF特征’;为此我们开发了匹配组成、分辨结合的验证协议以消除此类偏差。关键的是,我们超越相关性:通过在前向传播中消融字典方向,并测量模型自身预测分布的变化,证实特定特征是**因果性地**用于表示细胞类型特异的转录因子结合,而不仅仅是基序存在。在三种转录因子(CTCF、GATA1、REST)及两种架构下,因果验证的结合特征均稳定重现(每条件7–14/15个),而两类负控(打乱结合标签、随机选取特征)则无显著信号。该框架完全基于计算,仅使用公开数据,为基因组深度学习的可解释性声明提供可复用标准。

原文摘要 · Abstract (English)

Genomic language models achieve strong performance across regulatory-genomics tasks, yet what these models internally represent remains opaque, and the field lacks a principled procedure for verifying that an apparent ``concept'' inside a model is real rather than an artifact of sequence composition. We introduce a framework that combines sparse dictionary learning with causal intervention to extract, validate, and causally test interpretable features in genomic foundation models. Training top-$k$ sparse autoencoders on the hidden activations of two architecturally distinct models, Nucleotide Transformer ($6$-mer tokenization) and DNABERT-2 (byte-pair encoding), we recover thousands of monosemantic features that map to transcription-factor (TF) sequence motifs. We show that the naive validation of such features against position weight matrices is severely confounded by GC composition and repetitive elements, producing hundreds of spurious ``TF features'', and we develop a composition-matched, binding-resolved protocol that removes these confounds. Critically, we move beyond correlation: by ablating individual dictionary directions during the model's forward pass and measuring the induced shift in the model's own predictive distribution, we establish that specific features are \emph{causally} used to represent cell-type-specific TF binding, not merely motif presence. Across three transcription factors (CTCF, GATA1, REST) and both architectures, causally validated binding features emerge reproducibly ($7$--$14$ of $15$ tested features per condition), while two classes of negative control, scrambled binding labels and randomly selected features, yield no detectable signal. The framework is purely computational, uses only public data, and provides a reusable standard for interpretability claims in genomic deep learning.

基因组模型可解释性因果推理转录因子

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。