arXiv:2607.15623stat.MLcs.LG2026-07

提出稳定信号原则,解释模型重训练为何能收敛。

Retraining Seeks Stable Signals

  • 假设预测目标存在不变的稳定信号,重训练会收敛到该信号方向。
  • 即使模型影响远大于稳定信号,正则化仍使重训练几何收敛。
  • 适用于语言模型等数据反馈场景,揭示训练稳定性机制。

大规模部署的预测模型会影响未来数据,这种现象称为表演性(performativity)。应对方式通常是用新数据重新训练模型并重复部署,形成反馈循环。已有研究发现:当模型对数据的影响较小时,重训练可达到固定点。但固定点为何自然存在,以及强影响下重训练如何演化仍不明确。本文提出稳定信号原则,假设预测目标存在不依赖模型的微小成分(如物品内在质量),证明在非零稳定信号存在时,经过适当正则化的重训练将几何收敛至该信号方向,即便模型影响远大于稳定信号。正则化在此扮演控制表演性的自然角色,而非仅为了泛化。分析扩展至广义仿射重训练算子,涵盖任意模型引起的特征变化、异质时变效应及非线性响应。该视角亦适用于语言建模中的数据反馈,为模型生成数据下的训练稳定性提供新解释。

原文摘要 · Abstract (English)

Predictive models deployed at scale influence future data, a phenomenon called performativity. And there is always one way to cope: Train the model on new data, deploy it again, and repeat. This process, called retraining or repeated risk minimization, creates a feedback loop between model and data that real-world learning systems can't avoid. Results on performative prediction shed light on this dynamic: If the model's influence on the data is small, retraining reaches a fixed point. What remains open is why fixed points should naturally exist, and what governs retraining when the model's influence is strong. In this work we develop a new perspective on retraining -- the stable signal principle -- that addresses these questions. We start from the assumption that the prediction target has at least some small model-independent component, a stable signal, such as the intrinsic quality of an item. We prove that when a nonzero stable signal exists, repeated risk minimization, suitably regularized, converges geometrically to the direction of this stable signal. This is true even if the model's influence on the target is arbitrarily large relative to the stable signal. Regularization emerges naturally as a force to control performativity, rather than to promote generalization, revealing a new facet of an old concept. We extend the analysis to a broad family of affine retraining operators under arbitrary model-induced feature changes, heterogeneous time-varying effects, and nonlinear responses. The stable signal perspective also applies to data feedback loops in language modeling, providing new explanations for the stability of language model training from model-generated data.

机器学习反馈循环重训练稳定信号

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。