用累积分布函数归一化提升KAN模型预测能力
Improving KAN with CDF normalization to quantiles
- 将输入数据通过估计的CDF映射到[0,1]区间，实现近似分位数归一化
- 在Legendre-KAN上使用该方法后，预测性能显著提升
- 支持概率分布传播与反向传播，适合需要不确定性建模的任务
数据归一化在机器学习中至关重要，通常采用减均值除标准差或固定区间缩放。在金融领域的极值理论中，常用将数据转换为经验累积分布函数（CDF）值，使其近似服从[0,1]上的均匀分布，从而简化表示并降低过拟合风险。这一方法在机器学习中几乎未被关注。本文以近期流行的科尔莫戈罗夫-阿诺德网络（KAN）为例，仅通过将归一化方式从传统方法切换为基于估计CDF的分位数归一化，即刻显著提升了Legendre-KAN的预测表现。此外，在层次共现关系（HCR）解释下，此类神经元权重对应于局部联合分布的混合矩，可实现概率分布的传播，并支持传播方向的改变。
原文摘要 · Abstract (English)
Data normalization is crucial in machine learning, usually performed by subtracting the mean and dividing by standard deviation, or by rescaling to a fixed range. In copula theory, popular in finance, there is used normalization to approximately quantiles by transforming x to CDF(x) with estimated CDF (cumulative distribution function) to nearly uniform distribution in [0,1], allowing for simpler representations which are less likely to overfit. It seems nearly unknown in machine learning, therefore, we would like to present some its advantages on example of recently popular Kolmogorov-Arnold Networks (KANs), improving predictions from Legendre-KAN by just switching rescaling to CDF normalization. Additionally, in HCR interpretation, weights of such neurons are mixed moments providing local joint distribution models, allow to propagate also probability distributions, and change propagation direction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。