通过滑动窗口捕捉局部记忆信号，显著提升对微调大模型的成员推理攻击效果。
Window-based Membership Inference Attacks Against Fine-tuned Large Language Models
- 用滑动窗口和符号聚合方法，从局部文本片段中挖掘记忆痕迹。
- 在11个数据集上检测率提升2-3倍，低误报率下AUC显著更高。
- 适合关注大模型隐私安全的研究者或防御方案设计者。
针对大型语言模型（LLMs）的大多数成员推理攻击（MIAs）依赖全局信号（如平均损失）识别训练数据，但此类方法会弱化细微的局部记忆信号，降低攻击效果。本文挑战这一全局平均范式，提出局部上下文中的记忆信号更为显著。我们提出WBC（基于窗口的比较）方法，采用滑动窗口结合符号聚合策略：在不同大小的窗口内，通过目标模型与参考模型的损失对比进行二元投票，再以几何间隔分布的窗口尺寸进行集成。该方法可从词级特征到短语级结构均捕捉记忆模式。在11个数据集上的大量实验表明，WBC显著优于现有基线，在低误报率下检测率提升2-3倍，且AUC更高。研究揭示：聚合局部证据比全局平均更有效，暴露了微调后大模型的关键隐私漏洞。
原文摘要 · Abstract (English)
Most membership inference attacks (MIAs) against Large Language Models (LLMs) rely on global signals, like average loss, to identify training data. This approach, however, dilutes the subtle, localized signals of memorization, reducing attack effectiveness. We challenge this global-averaging paradigm, positing that membership signals are more pronounced within localized contexts. We introduce WBC (Window-Based Comparison), which exploits this insight through a sliding window approach with sign-based aggregation. Our method slides windows of varying sizes across text sequences, with each window casting a binary vote on membership based on loss comparisons between target and reference models. By ensembling votes across geometrically spaced window sizes, we capture memorization patterns from token-level artifacts to phrase-level structures. Extensive experiments across eleven datasets demonstrate that WBC substantially outperforms established baselines, achieving higher AUC scores and 2-3 times improvements in detection rates at low false positive thresholds. Our findings reveal that aggregating localized evidence is fundamentally more effective than global averaging, exposing critical privacy vulnerabilities in fine-tuned LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。