arXiv:2502.11411cs.LG2025-02被引 10

通过去噪表示提升危险数据检测精度,避免误判安全样本。

Detecting and Filtering Unsafe Training Data via Data Attribution with Denoised Representation

  • 基于隐藏状态/梯度相似性,用去噪表示定位危险训练数据
  • 在越狱攻击和性别偏见检测中显著优于现有方法
  • 适合需要高精度数据清洗的模型开发团队

大语言模型对少量危险训练数据极为敏感,因此有效的检测与过滤对构建可信模型至关重要。当前最先进的检测方法多依赖需大量计算的分类器,且仅限于预定义类别。本文探索基于表示的数据归因方法,通过测量训练样本与少量危险目标样本之间的相似性(如隐藏状态或梯度)。我们发现:危险文本包含关键有害词与必要中性词(如停用词或无害事实),后者使整体表示“噪声”化,影响检测效果。为此,提出去噪表示归因(DRA)方法,对训练数据和目标表示进行去噪处理,提升归因准确性。在越狱攻击过滤与性别偏见检测任务中,该方法显著优于主流基于分类器的SOTA方法。

原文摘要 · Abstract (English)

Large language models (LLMs) are highly sensitive to even small amounts of unsafe training data, making effective detection and filtering essential for trustworthy model development. Current state-of-the-art (SOTA) detection approaches primarily rely on moderation classifiers, which require significant computation overhead for training and are limited to predefined taxonomies. In this work, we explore data attribution approaches that measure the similarity between individual training samples and a small set of unsafe target examples, based on data representations such as hidden states or gradients. We identify a key limitation in existing methods: unsafe target texts contain both critical tokens that make them unsafe and neutral tokens (e.g., stop words or benign facts) that are necessary to form fluent language, and the latter of which makes the overall representations ``noisy'' for the purpose of detecting unsafe training data. To address this challenge, we propose Denoised Representation Attribution (DRA), a novel representation-based data attribution approach that denoises training and target representations for unsafe data detection. Across tasks of filtering jailbreaks and detecting gender bias, the proposed approach leads to significant improvement for data attribution methods, outperforming SOTA methods that are mostly based on moderation classifiers.

数据清洗安全检测表示去噪

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。