arXiv:2605.17493cs.LGcs.AI2026-05被引 1

用非线性激活提升天气模型可解释性,发现隐藏的气候特征

Beyond Linear Superposition: Discovering Climate Features in AI Weather Models with KAN-SAE

论文配图:Beyond Linear Superposition: Discovering Climate Features in AI Weather Models with KAN-SAE
图 1 · 摘自论文原文
  • 用基于KAN的非线性激活替代传统线性编码,让每个特征自适应学习非线性门控
  • 在Sonny模型上发现975个有效特征,冗余降低20%,比线性基线多72%特征
  • 无气候标注下识别出欧洲热浪和西太平洋台风路径,经因果实验验证

深度学习天气预测模型具备出色预测能力,但其内部如何表征物理气候现象仍不透明。通过稀疏自编码器(SAEs)实现机制可解释性是条可行路径,但现有SAEs假设特征严格线性叠加,难以捕捉现代Transformer中高度非线性的大气动力学。本文提出KAN-SAE,其编码器将标准ReLU替换为来自柯尔莫哥洛夫-阿诺德网络(KANs)的可学习分段样条激活,使每个潜在维度可发展自身非线性门控。应用于Sonny模型时,KAN-SAE发现975个活跃特征(线性基线为566个,提升72%),特征间冗余降低20%,重建保真度相当。无需气候监督,即识别出空间集中于西欧的欧洲热浪特征,以及经因果引导实验验证的西太平洋台风追踪特征。结果表明,非线性激活对深度学习天气模型的机制可解释性至关重要,能恢复线性基线无法捕捉的气候特征。

原文摘要 · Abstract (English)

Deep learning weather prediction models achieve remarkable predictive skill yet remain largely opaque: we know little about how they represent physical climate phenomena internally. Mechanistic interpretability through Sparse Autoencoders (SAEs) offers a principled route to decomposing these representations, but existing SAEs assume strictly linear feature superposition - a constraint ill-suited for the highly nonlinear atmospheric dynamics encoded in modern transformers. We introduce KAN-SAE, a sparse autoencoder whose encoder replaces the standard ReLU with learnable per-feature B-spline activations drawn from Kolmogorov-Arnold Networks (KANs), allowing each latent dimension to develop its own nonlinear gating profile. Applied to Sonny, KAN-SAE discovers 975 alive features (vs. 566 for a linear baseline, a 72% improvement) with 20% lower inter-feature redundancy and comparable reconstruction fidelity. Without any climate supervision, KAN-SAE identifies an interpretable European heatwave feature spatially concentrated over western Europe, and a western Pacific typhoon tracker confirmed by causal steering experiments. Our results demonstrate that nonlinear activations are essential for mechanistic interpretability of deep learning weather prediction models, recovering climate features that remain invisible to linear baselines.

可解释性气候建模非线性网络自编码器

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。