arXiv:2512.01514cs.LG2025-12

破解黑箱文本分类器的标签语义,让未知标签可解释。

Label Forensics: Interpreting Hard Labels in Black-Box Text Classifier

  • 用句子嵌入分布建模标签语义,通过迭代采样优化代表性种子句。
  • 在多个黑箱模型上实现约92.24%的标签一致性,精准还原分类概念。
  • 适合模型审计、可信AI研究者,尤其适用于无文档的HuggingFace模型。

自然语言处理技术的广泛应用催生了大量文本分类器,但许多模型内部语义未公开甚至刻意隐藏。这类仅提供硬标签输出的黑箱模型可能在不受监管的网络环境中运行,或被用于未知目的,引发合法审计与溯源关切。本文提出标签取证(Label Forensics),一种黑箱框架,旨在推断未文档化分类器中各标签所代表的语义概念。具体地,我们通过句子嵌入分布来表征标签,其中任意样本均能可靠反映分类器对该标签隐式学习到的概念。该分布需满足两个关键特性:精确性(样本一致归入目标标签)与泛化性(覆盖标签的广泛语义空间)。为此,我们设计了语义邻域采样器与迭代优化流程,选取一组联合最大化标签一致性和分布覆盖率的代表性种子句。最终输出为优化后的种子句集与采样器,构成标签语义的实证分布。在多个黑箱分类器上的实验显示,平均标签一致性达92.24%,证明嵌入区域能准确捕捉分类器的标签语义。我们在一个未文档化的HuggingFace分类器上进一步验证框架,实现细粒度标签解读,支持负责任的AI审计。

原文摘要 · Abstract (English)

The widespread adoption of natural language processing techniques has led to an unprecedented growth of text classifiers across the modern web. Yet many of these models circulate with their internal semantics undocumented or even intentionally withheld. Such opaque classifiers, which may expose only hard-label outputs, can operate in unregulated web environments or be repurposed for unknown intents, raising legitimate forensic and auditing concerns. In this paper, we position ourselves as investigators and work to infer the semantic concept each label encodes in an undocumented black-box classifier. Specifically, we introduce label forensics, a black-box framework that reconstructs a label's semantic meaning. Concretely, we represent a label by a sentence embedding distribution from which any sample reliably reflects the concept the classifier has implicitly learned for that label. We believe this distribution should maintain two key properties: precise, with samples consistently classified into the target label, and general, covering the label's broad semantic space. To realize this, we design a semantic neighborhood sampler and an iterative optimization procedure to select representative seed sentences that jointly maximize label consistency and distributional coverage. The final output, an optimized seed sentence set combined with the sampler, constitutes the empirical distribution representing the label's semantics. Experiments on multiple black-box classifiers achieve an average label consistency of around 92.24 percent, demonstrating that the embedding regions accurately capture each classifier's label semantics. We further validate our framework on an undocumented HuggingFace classifier, enabling fine-grained label interpretation and supporting responsible AI auditing.

黑箱分析标签解释语义还原AI审计

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。