闭环学习中模型参数会放大数据初始偏差，但引入真实数据可避免此问题。
Lost in Retraining: Roaming the Parameter Space of Exponential Families Under Closed-Loop Learning
- 用指数族模型推导参数演化方程，揭示闭环学习动态机制。
- 最大似然估计使充分统计量具鞅性质，导致参数收敛至放大偏差的吸收态。
- 只要有真实数据点，用最大后验或正则化就能阻止偏差放大，适合关注自训练安全的研究者。
闭环学习指反复从模型自身生成的数据中估计模型的过程，未来大模型可能主要依赖神经网络生成的数据进行训练。本文研究属于指数族的模型在该过程中的参数演化，推导出控制参数动态的微分方程。结果表明，最大似然估计使充分统计量具备鞅性质，导致系统收敛至吸收态，从而放大初始数据中的偏差。然而，若数据中至少包含一个来自真实模型的数据点，则可通过最大后验估计或引入正则化来避免这一问题。
原文摘要 · Abstract (English)
Closed-loop learning is the process of repeatedly estimating a model from data generated from the model itself. It is receiving great attention due to the possibility that large neural network models may, in the future, be primarily trained with data generated by artificial neural networks themselves. We study this process for models that belong to exponential families, deriving equations of motions that govern the dynamics of the parameters. We show that maximum likelihood estimation of the parameters endows sufficient statistics with the martingale property and that as a result the process converges to absorbing states that amplify initial biases present in the data. However, we show that this outcome may be prevented if the data contains at least one data point generated from a ground truth model, by relying on maximum a posteriori estimation or by introducing regularisation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。