提出快慢适应框架,实现少样本跨域预测。
Adapting, Fast and Slow: On Few-Shot Transportability of Compositions
- 用因果机制组合构建可迁移模型,支持零样本预测。
- 仅需少量目标域数据,即可实现低误差少样本适配。
- 适合需要快速跨域部署的场景,如医疗诊断、工业检测。
跨域泛化依赖于连接源域与目标域分布的稳定结构。基于因果可迁移性理论,我们研究一种序列预测设置,其中目标预测器可表示为由源数据中可学习的因果机制组成的电路。引入两类可迁移性:模块可迁移性描述原子情形,即目标预测器由单一源域可学习机制构成;电路可迁移性将其扩展至由多个源域模块组合而成的目标预测器,即使无直接预测目标标签的源机制,也能实现零样本预测。我们在逐步放松假设条件下研究这些电路。首先,在已知源域和目标域因果结构的前提下,给出仅从源数据学习相关电路的条件。随后放宽结构假设,允许少量目标域数据。特别地,提出一种无需显式因果结构的监督域适应方案,学习电路并获得少样本保证——可达到的误差与能从源域模块组合而成的最小目标电路大小相关。最后,提出基于梯度的符号电路搜索松弛方法,并实证验证其能定性捕捉到有/无中间过程监督下的快速适应与无匹配源机制时的慢适应规律。
原文摘要 · Abstract (English)
Generalization across domains requires stable structure that links the source and target distributions. Building on causal transportability theory, we study a sequential prediction setting in which the target predictor can be represented as a circuit composed of causal mechanisms that are learnable from source data. We introduce two classes of transportability. Module transportability captures the atomic case, where the target predictor is given by a mechanism learnable from a single source domain. Circuit transportability generalizes this idea to target predictors obtained by composing several modules learned from source data, enabling zero-shot prediction even when no source mechanism directly predicts the target label. We study these classes of circuits under increasingly relaxed assumptions. First, we provide conditions under which the relevant circuits can be learned from source data alone, given causal knowledge about the source and target domains. We then relax these structural assumptions by allowing limited data from the target domain. In particular, we develop a supervised domain adaptation scheme that learns circuits without requiring explicit causal structure. The resulting few-shot guarantees tie the achievable error to the size of the smallest target circuit composable from modules learned from source data. Finally, we propose a gradient-based relaxation of the symbolic circuit search and evaluate it empirically, showing that it qualitatively tracks the predicted regimes of fast adaptation -- with and without process supervision over intermediate positions -- and slow adaptation when no source mechanism matches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。