量化条件神经过程的预测不一致性，揭示其在少样本时的潜在风险。
On the Conditioning Consistency Gap in Conditional Neural Processes
- 提出条件一致性差距（KL散度）衡量添加点与条件化时预测差异。
- 证明一致差距随上下文大小n呈1/n²衰减，且该速率紧致。
- 适用于关注少样本泛化性与理论可靠性研究的机器学习从业者。
神经过程是将上下文集映射为预测分布的元学习模型。尽管受随机过程启发，神经过程通常不满足定义有效随机过程所需的柯尔莫哥洛夫一致性条件。这一不一致虽被广泛承认，但理解不足。实践者注意到神经过程表现良好，却未量化其含义。本文通过定义条件一致性差距（即条件神经过程在添加点与条件化时预测变化的KL散度），填补此空白。主要结果表明：对于有界编码器和Lipschitz解码器的条件神经过程，一致性差距为O(1/n²)，且该速率紧致。这些界精确刻画了神经过程对有效随机过程的逼近程度。不一致在中等上下文规模下可忽略，但在少样本情形下可能显著。
原文摘要 · Abstract (English)
Neural processes are meta-learning models that map context sets to predictive distributions. While inspired by stochastic processes, NPs do not generally satisfy the Kolmogorov consistency conditions required to define a valid stochastic process. This inconsistency is widely acknowledged but poorly understood. Practitioners note that NPs work well despite the violation, without quantifying what this means. We address this gap by defining the conditioning consistency gap, a KL divergence measuring how much a conditional neural process's (CNP) predictions change when a point is added to the context versus conditioned upon. Our main results show that for CNPs with bounded encoders and Lipschitz decoders, the consistency gap is $O(1/n^2)$ in context size $n$, and that this rate is tight. These bounds establish the precise sense in which CNPs approximate valid stochastic processes. The inconsistency is negligible for moderate context sizes but can be significant in the few-shot regime.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。