让模型区分物理信号与仪器干扰,提升小样本下的预测能力
Causal Foundation Models: Disentangling Physics from Instrument Properties
- 用双编码器+对比学习分离物理信号和仪器影响
- 在低数据场景下预测性能显著优于传统模型
- 适合需要跨仪器泛化的时序数据研究者
结构化时间序列的基础模型面临根本挑战:观测数据常将真实物理现象与测量仪器引入的系统性畸变混杂在一起,限制了模型在异构或多仪器场景下的泛化能力。本文提出一种因果驱动的基础模型,采用双编码器架构并结合结构化对比学习,显式分离物理因素与仪器效应。利用自然存在的观测三元组(同一目标在不同条件下测量,或不同目标在相同条件下测量),模型学习到物理信号与仪器影响的独立潜在表示。在模拟的天文时间序列数据上评估,这些数据模拟了NASA凌日系外行星巡天卫星(TESS)观测变星的复杂性,本方法在下游预测任务中显著优于传统单潜空间基础模型,尤其在低数据情况下表现更优。结果表明,该模型具备基础模型的关键能力,如少样本泛化与高效适应,并强调在结构化数据表示学习中编码因果结构的重要性。
原文摘要 · Abstract (English)
Foundation models for structured time series data must contend with a fundamental challenge: observations often conflate the true underlying physical phenomena with systematic distortions introduced by measurement instruments. This entanglement limits model generalization, especially in heterogeneous or multi-instrument settings. We present a causally-motivated foundation model that explicitly disentangles physical and instrumental factors using a dual-encoder architecture trained with structured contrastive learning. Leveraging naturally occurring observational triplets (i.e., where the same target is measured under varying conditions, and distinct targets are measured under shared conditions) our model learns separate latent representations for the underlying physical signal and instrument effects. Evaluated on simulated astronomical time series designed to resemble the complexity of variable stars observed by missions like NASA's Transiting Exoplanet Survey Satellite (TESS), our method significantly outperforms traditional single-latent space foundation models on downstream prediction tasks, particularly in low-data regimes. These results demonstrate that our model supports key capabilities of foundation models, including few-shot generalization and efficient adaptation, and highlight the importance of encoding causal structure into representation learning for structured data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。