改进标签处理让核模型在噪声数据中表现超越神经网络
K-Inverse-RFM: A Modified RFM that Bridges the Gap to Neural Networks for Data-Corrupted Mathematical Tasks
- 对训练标签进行特定变换，增强模型抗噪能力
- 在复杂表示和类别不平衡数据上性能接近甚至超过神经网络
- 方法简单但效果显著，适合处理低质量数学任务数据
递归特征机器（RFM）是一类利用平均梯度外积（AGOP）进行特征学习的核方法。研究表明，其在多种场景下能有效复现前馈神经网络（FNN）的学习动态与特征表示。然而，在某些数据污染场景中，尽管特征学习能力相当，RFM的性能仍显著低于神经网络。本文针对数学问题中的这一局限展开研究，提出一种对训练标签的巧妙变换，显著提升模型在噪声、复杂表示及类别不平衡数据中的学习能力。该简单而强大的调整使RFM在多个场景中缩小甚至超越了与FNN的性能差距。
原文摘要 · Abstract (English)
Recursive Feature Machines (RFMs) are a class of kernel machines that utilize the Average Gradient Outer Product (AGOP) as a mechanism for feature learning. They have been shown to effectively replicate the learning dynamics and feature representations of Feedforward Neural Networks (FNNs) across various settings. However, despite comparable capacity for feature learning and the similarities in the features they acquire, RFMs exhibit significantly lower performance than neural networks in certain data-corrupted scenarios. In this work, we investigate these limitations in mathematical problems. As a solution, we introduce a remarkably effective transformation applied to the training labels which promotes learning in noisy, complexly represented, and class-imbalanced data. This simple yet powerful adjustment enables RFMs to close the performance gap with FNNs and, in some cases, even surpass them.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。