用有向无环图约束Transformer，提升模型可解释性与鲁棒性。
CaTs and DAGs: Integrating Directed Acyclic Graphs with Transformers for Causally Constrained Predictions
- 在Transformer中引入有向无环图（DAG）作为因果约束框架
- 保持强大拟合能力的同时，确保预测符合预设因果结构
- 适合需要可解释性与鲁棒性的实际应用，如医疗、金融
人工神经网络（ANNs），包括全连接网络和Transformer，在计算机视觉与自然语言处理等领域广泛应用。然而，它们无法天然地遵守因果结构，导致在分布外数据上表现脆弱，且难以解释，限制了其在真实场景中的可靠性。本文提出因果Transformer（CaTs），一种基于预定义有向无环图（DAG）的通用模型类，可在遵循特定因果约束的前提下运行。CaTs 保留了传统神经网络的强大函数逼近能力，同时在推理时满足底层结构约束，从而提升了模型的鲁棒性、可靠性与可解释性。该方法为在对可靠性与可解释性要求高的真实场景中部署神经网络开辟了新路径。
原文摘要 · Abstract (English)
Artificial Neural Networks (ANNs), including fully-connected networks and transformers, are highly flexible and powerful function approximators, widely applied in fields like computer vision and natural language processing. However, their inability to inherently respect causal structures can limit their robustness, making them vulnerable to covariate shift and difficult to interpret/explain. This poses significant challenges for their reliability in real-world applications. In this paper, we introduce Causal Transformers (CaTs), a general model class designed to operate under predefined causal constraints, as specified by a Directed Acyclic Graph (DAG). CaTs retain the powerful function approximation abilities of traditional neural networks while adhering to the underlying structural constraints, improving robustness, reliability, and interpretability at inference time. This approach opens new avenues for deploying neural networks in more demanding, real-world scenarios where robustness and explainability is critical.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。