发现离散扩散语言模型易受成员推理攻击，隐私风险远高于此前估计。
Membership Inference Attacks on Discrete Diffusion Language Models
- 从四个掩码比例的重建损失提取46维特征，用XGBoost和MLP分类器攻击
- 在MIMIR上平均AUC达0.878，最高0.930，比基线高0.062
- 仅用ELBO轨迹即可实现近似全性能，无需注意力特征
掩码扩散语言模型（MDLMs）以迭代去掩码替代自回归生成，但其隐私特性尚未充分研究。本文研究微调后的MDLMs上的成员推理攻击（MIA），发现其脆弱性远超现有灰盒基线。通过在四个掩码比例下提取46维重建损失特征，训练XGBoost与MLP分类器。在涵盖六个文本领域的MIMIR基准上，XGBoost平均AUC达0.878，峰值0.930（在Pile CC上），较SAMA灰盒基线平均提升0.062。留一信号消融实验表明，移除ELBO轨迹导致平均下降0.130，而注意力特征贡献不足0.003。此外，设计了阴影模型迁移攻击：仅用3个来自无关领域的代理MDLMs生成标签，无需访问目标域数据，仍达到0.858平均AUC，仅比白盒基准低0.020，证明该攻击路径实用且高效。
原文摘要 · Abstract (English)
Masked Diffusion Language Models MDLMs replace autoregressive generation with iterative demasking and their privacy properties are largely unstudied. We study membership inference attacks MIA on fine tuned MDLMs and show they are significantly more vulnerable than current grey box baselines suggest. We extract a 46 dimensional feature vector from the models reconstruction loss at four masking ratios and train XGBoost and MLP classifiers on top. On the MIMIR benchmark across six text domains XGBoost achieves mean AUC 0.878 peaking at 0.930 on Pile CC and beats the SAMA grey box baseline by 0.062 AUC on average. A leave one signal out ablation shows that the ELBO trajectory alone drives most of this with a mean drop of 0.130 when removed while attention features add almost nothing below 0.003. We also design a shadow model transfer attack where K equals 3 surrogate MDLMs trained on data from unrelated domains generate classifier labels with no access to the target domain. This achieves 0.858 mean AUC within 0.020 of the white box oracle and establishes shadow model transfer as a practical and near equally effective attack path.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。