揭示了矩形流模型的记忆机制并提出新防御方法
Generalization and Memorization in Rectified Flow
- 设计复杂度校准的隐私攻击指标,精准区分记忆与图像本身复杂度
- 发现模型在训练中点时最易被隐私攻击,攻击准确率提升45%
- 用U型时间采样替代均匀采样,降低记忆风险且不损失生成质量
基于流匹配的矩形流(Rectified Flow)已成为高效高保真图像生成的主流范式。然而,现有研究多关注生成质量与模型规模,对模型如何记忆训练数据的动态机制仍缺乏深入探索。本文通过会员推理攻击(MIA)测试统计量系统研究矩形流的记忆行为,逐步提出三种测试统计量,最终构建出能解耦图像空间复杂度与真实记忆信号的复杂度校准指标。该指标使攻击AUC最高提升15%,隐私关键指标TPR@1%FPR最高提升45%,首次为矩形流设计出非平凡的隐私攻击方法。基于此,我们发现标准均匀时间训练下,模型在积分中点时对隐私攻击最为敏感,这由网络对线性近似的强制偏离所解释。进一步证明,采用对称指数分布(U型)的时间采样可有效减少暴露于脆弱中间时间步的风险。在三个数据集上的大量实验表明,该时间正则化在抑制记忆的同时保持了生成保真度。
原文摘要 · Abstract (English)
Generative models based on the Flow Matching objective, particularly Rectified Flow, have emerged as a dominant paradigm for efficient, high-fidelity image synthesis. However, while existing research heavily prioritizes generation quality and architectural scaling, the underlying dynamics of how RF models memorize training data remain largely underexplored. In this paper, we systematically investigate the memorization behaviors of RF through the test statistics of Membership Inference Attacks (MIA). We progressively formulate three test statistics, culminating in a complexity-calibrated metric that successfully decouples intrinsic image spatial complexity from genuine memorization signals. This calibration yields a significant performance surge -- boosting attack AUC by up to 15% and the privacy-critical TPR@1%FPR metric by up to 45% -- establishing the first non-trivial MIA specifically tailored for RF. Leveraging these refined metrics, we uncover a distinct temporal pattern: under standard uniform temporal training, a model's susceptibility to MIA strictly peaks at the integration midpoint, a phenomenon we justify via the network's forced deviation from linear approximations. Finally, we demonstrate that substituting uniform timestep sampling with a Symmetric Exponential (U-shaped) distribution effectively minimizes exposure to vulnerable intermediate timesteps. Extensive evaluations across three datasets confirm that this temporal regularization suppresses memorization while preserving generative fidelity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。