突破线性假设,让机器学习模型在数据分布变化时仍稳定预测。
Nonlinear Performative Prediction
- 用核方法拓展非线性场景下的预测模型设计
- 通过误差差异量化数据分布偏移,实现稳定预测
- 适合关注实际应用中模型鲁棒性的研究者
Performative prediction 是一种新兴的机器学习范式,应对模型预测导致数据分布变化的场景。现有工作多依赖不可控假设(如损失梯度有界),且仅限于线性案例,难以反映真实世界复杂非线性特性。本文放松这些限制,提出基于最大间隔的损失函数,结合核方法推广至非线性空间。采用预测误差在两个分布间的差异作为分布偏移的度量,推导出线性和非线性情形下 performative 稳定性的充要条件。据此设计算法,保证模型稳定性。在合成与真实数据集上的实验验证了方法在非线性场景下的优越性能,优于当前最优基线。
原文摘要 · Abstract (English)
Performative prediction is an emerging paradigm in machine learning that addresses scenarios where the model's prediction may induce a shift in the distribution of the data it aims to predict. Current works in this field often rely on uncontrollable assumptions, such as bounded gradients of performative loss, and primarily focus on linear cases in their examples and evaluations to maintain consistency between theoretical guarantees and empirical validations. However, such linearity rarely holds in real-world applications, where the data usually exhibit complex nonlinear characteristics. In this paper, we relax these out-of-control assumptions and present a novel design that generalizes performative prediction to nonlinear cases while preserving essential theoretical properties. Specifically, we formulate the loss function of performative prediction using a maximum margin approach and extend it to nonlinear spaces through kernel methods. To quantify the data distribution shift, we employ the discrepancy between prediction errors on these two distributions as an indicator, which characterizes the impact of the performative effect on specific learning tasks. By doing so, we can derive, for both linear and nonlinear cases, the conditions for performative stability, a critical and desirable property in performative contexts. Building on these theoretical insights, we develop an algorithm that guarantees the performative stability of the predictive model. We validate the effectiveness of our method through experiments on synthetic and real-world datasets with both linear and nonlinear data distributions, demonstrating superior performance compared to state-of-the-art baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。