提出新指标CPCF，用置信度变化量化持续学习中的灾难性遗忘。
A Conformal Predictive Measure for Assessing Catastrophic Forgetting
- 基于自适应置信区间，通过模型对旧任务的置信度下降衡量遗忘程度。
- 在4个基准数据集上，CPCF与旧任务准确率高度相关，验证其有效性。
- 适合需要实时监测遗忘的场景，如在线学习、机器人持续训练。
本文提出一种评估持续学习中灾难性遗忘（CF）的新方法。我们引入基于置信区间（CP）的度量——置信因子（CPCF），用于有效量化和评估遗忘现象。该框架利用自适应置信区间技术，通过监控模型对先前任务预测置信度的变化来估计遗忘程度。此方法为在引入新任务时动态、实用地监测和评估旧任务性能提供了支持，更适用于真实应用场景。在四个基准数据集上的实验结果表明，CPCF与旧任务准确率之间存在强相关性，验证了该度量的可靠性和可解释性。研究结果凸显了CPCF在动态学习环境中评估和理解灾难性遗忘的潜力。
原文摘要 · Abstract (English)
This work introduces a novel methodology for assessing catastrophic forgetting (CF) in continual learning. We propose a new conformal prediction (CP)-based metric, termed the Conformal Prediction Confidence Factor (CPCF), to quantify and evaluate CF effectively. Our framework leverages adaptive CP to estimate forgetting by monitoring the model's confidence on previously learned tasks. This approach provides a dynamic and practical solution for monitoring and measuring CF of previous tasks as new ones are introduced, offering greater suitability for real-world applications. Experimental results on four benchmark datasets demonstrate a strong correlation between CPCF and the accuracy of previous tasks, validating the reliability and interpretability of the proposed metric. Our results highlight the potential of CPCF as a robust and effective tool for assessing and understanding CF in dynamic learning environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。