arXiv:2604.23987cs.LG2026-04

大模型持续学习中,可靠性会先于准确率崩溃,需额外校准。

Continual Calibration: Coverage Can Collapse Before Accuracy in Lifelong LLM Fine-Tuning

  • 提出校准重放法,用小缓冲区动态调整置信阈值。
  • 实测覆盖度损失是准确率损失的3.4倍,最高降17%仍保精度。
  • 适合关注模型可信度的持续学习研究者,尤其分类任务。

大型语言模型的持续学习通常以序列微调下的准确率保留来评估。我们指出这一视角不完整,因为不确定性可靠性可能在顶级-1性能下降前更早、更剧烈地退化。通过在三种模型族和八个任务序列(主要来自分类与选择题基准)上测量共形覆盖度与校准误差,我们发现:在所研究的分类场景中,覆盖度损失平均比准确率损失高出约3.4倍(±0.5倍);最严重情况下,覆盖度从0.92降至0.61,而准确率仅比基线低3个百分点以内。标准持续学习方法虽能保持准确率,但无法自动维持覆盖度;简单校准基线也仅部分恢复差距。为此,我们提出校准重放(calibration replay),一种轻量级后处理方法:维护任务特异性预留缓冲区,在每次更新后重新拟合任务特定的共形阈值。该方法无需训练时梯度开销,内存占用不足普通经验回放的1%,在缓冲区大小m=200时,通常可将覆盖度恢复至名义值的±2点以内。我们辅以漂移分解、有限样本恢复定理(在可交换性下保证精确共形有效性)及混合有效性命题,解释为何聚合阈值无效。理论保证适用于具有任务特异性缓冲区的分类任务;生成任务的扩展为探索性工作。

原文摘要 · Abstract (English)

Continual learning for large language models is typically evaluated through accuracy retention under sequential fine-tuning. We argue that this perspective is incomplete, because uncertainty reliability can degrade earlier and more sharply than top-1 performance. We study this empirically by measuring conformal coverage and calibration error on sequentially fine-tuned models across three model families and eight task sequences drawn primarily from classification and multiple-choice benchmarks. Across the classification-style settings we study, coverage loss exceeds accuracy loss by a factor of roughly \(3.4\times \pm 0.5\times\) on average across seeds; in the most pronounced case, coverage drops from \(0.92\) to \(0.61\), while accuracy remains within three points of baseline. Standard continual-learning methods that preserve accuracy do not automatically preserve coverage, and naive calibration baselines recover only part of the gap. We propose calibration replay, a lightweight post-hoc procedure that maintains a task-specific held-out buffer and refits a task-specific conformal threshold under the current model after each update. It adds no training-time gradient cost, uses less than one percent of the memory of ordinary experience replay, and typically restores coverage to within two points of nominal at buffer size \(m = 200\). We accompany the empirical study with a drift decomposition, a finite-sample recovery theorem showing exact conformal validity under exchangeability, and a mixture-validity proposition explaining why pooled thresholds do not suffice. Our guarantees are stated for classification-style tasks with task-specific buffers; extensions to open-ended generation are exploratory.

持续学习校准可靠性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。