提出可评估元学习中数据贡献的通用框架,解决数据噪声与低效问题。
Evaluating Data Influence in Meta Learning
- 基于双层优化设计任务级与实例级影响函数,闭环分析数据作用
- 在多个下游任务上验证了对噪声数据和低贡献任务的有效识别能力
- 适合关注元学习训练质量、数据清洗或模型可解释性的研究者
元学习作为解决少样本学习的核心方法,仍面临训练数据效率低下和标签噪声等问题。由于其双层结构中元参数与任务特定参数相互依赖,现有数据影响评估工具难以准确应用。为此,本文基于影响函数,在双层优化框架下提出通用的数据归属评估框架。该框架引入任务影响函数(task-IF)和实例影响函数(instance-IF),以闭式解精确衡量特定任务和单个数据点的影响。该方法全面建模了数据在内层与外层训练过程中的直接与间接贡献,涵盖数据对元参数的直接影响及其通过任务参数的传导效应。同时提供了多项提升计算效率与可扩展性的策略。实验结果表明,该框架在多个下游任务中有效实现了训练数据评估。
原文摘要 · Abstract (English)
As one of the most fundamental models, meta learning aims to effectively address few-shot learning challenges. However, it still faces significant issues related to the training data, such as training inefficiencies due to numerous low-contribution tasks in large datasets and substantial noise from incorrect labels. Thus, training data attribution methods are needed for meta learning. However, the dual-layer structure of mata learning complicates the modeling of training data contributions because of the interdependent influence between meta-parameters and task-specific parameters, making existing data influence evaluation tools inapplicable or inaccurate. To address these challenges, based on the influence function, we propose a general data attribution evaluation framework for meta-learning within the bilevel optimization framework. Our approach introduces task influence functions (task-IF) and instance influence functions (instance-IF) to accurately assess the impact of specific tasks and individual data points in closed forms. This framework comprehensively models data contributions across both the inner and outer training processes, capturing the direct effects of data points on meta-parameters as well as their indirect influence through task-specific parameters. We also provide several strategies to enhance computational efficiency and scalability. Experimental results demonstrate the framework's effectiveness in training data evaluation via several downstream tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。