arXiv:2502.07369stat.MLcs.LG2025-02

无需测试数据,就能统一评估不同特征在多种任务下的泛化能力。

Uniform Kernel Prober

  • 用核函数定义的伪度量,统一比较不同特征表示的性能。
  • 仅需n个样本即可高效估计,误差随1/√n衰减。
  • 适合研究特征泛化性,尤其对多任务学习模型有指导意义。

在多任务学习中,能否从训练数据中识别出在多个下游任务上均能实现低预测误差的有用特征,是成功的关键。然而实践中常面临预测任务选择和测试数据不可得的问题。本文提出一类称为统一核探测器(Uniform Kernel Prober, UKP)的伪度量,用于在不依赖测试数据的前提下,比较不同统计模型(如神经网络)所学特征在核岭回归任务中的表现。UKP 在给定核函数下,提供了一类广义核岭回归任务上的统一预测误差度量。通过合理选择核函数,可有效捕捉表示所需的不变性特性;且仅需 n 个输入样本,即可以 O(1/√n) 的误差率高效估计该伪度量。实验表明,UKP 能有效区分不同类型特征在下游核岭回归任务上的泛化性能差异。

原文摘要 · Abstract (English)

The ability to identify useful features or representations of the input data based on training data that achieves low prediction error on test data across multiple prediction tasks is considered the key to multitask learning success. In practice, however, one faces the issue of the choice of prediction tasks and the availability of test data from the chosen tasks while comparing the relative performance of different features. In this work, we develop a class of pseudometrics called Uniform Kernel Prober (UKP) for comparing features or representations learned by different statistical models such as neural networks when the downstream prediction tasks involve kernel ridge regression. The proposed pseudometric, UKP, between any two representations, provides a uniform measure of prediction error on test data corresponding to a general class of kernel ridge regression tasks for a given choice of a kernel without access to test data. Additionally, desired invariances in representations can be successfully captured by UKP only through the choice of the kernel function and the pseudometric can be efficiently estimated from $n$ input data samples with $O(\frac{1}{\sqrt{n}})$ estimation error. We also experimentally demonstrate the ability of UKP to discriminate between different types of features or representations based on their generalization performance on downstream kernel ridge regression tasks.

多任务学习特征评估核方法泛化性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。