arXiv:2602.01135cs.LG2026-02被引 2

用自回归模型直接挖掘数据中的因果关系,无需重新训练。

Your Autoregressive Model Already Reveals the Causal Graph

  • 利用预训练自回归模型估计条件独立性,构建因果图。
  • 在真实车辆诊断数据上,比最强基线高20多的F1分数。
  • 适合处理大规模、长序列的单流数据,如设备日志和医疗轨迹。

通过下一个词预测训练的自回归模型,隐式学习了数据生成过程的条件独立结构。我们利用这一观察,从单一离散事件序列中实现可扩展的因果发现,无需任何任务特定的重训练。此类单流场景常见于车辆诊断、制造系统和患者病程记录,但因缺乏重复样本、词汇量巨大及长程时间依赖,现有方法要么不准确,要么计算不可行。我们提出TRACE框架,将任意预训练自回归模型重用于条件互信息的密度估计,这是条件独立性检验的基本单元。通过在GPU上并行化构建条件独立测试,TRACE可线性扩展于词汇量大小,同时自然处理延迟因果效应。关键的是,我们证明最小化标准交叉熵预训练损失,实际上最小化了因果识别误差的上界,建立了序列预测与因果发现之间的对偶性。在非线性结构因果模型(|X|=8000)和真实车辆诊断日志(|X|=29100)上,TRACE是首个适用于该规模的方法,性能超过最强基线20多个F1点。

原文摘要 · Abstract (English)

Autoregressive models trained via next-token prediction implicitly learn the conditional independence structure of their data-generating process. We exploit this observation to perform scalable causal discovery from a single observed sequence of discrete events -- without any task-specific retraining. Such single-stream settings arise naturally in vehicle diagnostics, manufacturing systems, and patient trajectories, yet they remain largely unsolved: the absence of repeated samples, massive event vocabularies, and long-range temporal dependencies render existing methods either inaccurate or computationally intractable. We introduce TRACE, a framework that repurposes any pretrained autoregressive model as a density estimator for conditional mutual information, the fundamental primitive for conditional independence testing. By constructing parallelized CI tests on GPUs, TRACE recovers both the sample-level time causal graph and its summary projection, scaling linearly with the vocabulary size while naturally handling delayed causal effects. Crucially, we prove that minimizing the standard cross-entropy pretraining loss directly minimizes an upper bound on the causal identification error, establishing a duality between sequence prediction and causal discovery. On nonlinear SCMs (|X| = 8000) and real-world vehicle diagnostic logs (|X| = 29100), TRACE is the first applicable method at this scale, outperforming the strongest baseline by over 20 F1 points.

因果发现自回归模型序列分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。