arXiv:2608.14649cs.LG2026-08中稿 · SIAM SDM 2026, 39 …

无需训练的文本多标签分类新方法,利用扩散模型概率判断标签真伪。

Discrete Diffusion Language Models Are Training-Free Multi-Label Classifiers

论文配图:Discrete Diffusion Language Models Are Training-Free Multi-Label Classifiers
图 1 · 摘自论文原文
  • 为每个标签生成是/否问题,通过掩码位置概率比较判定标签归属。
  • 在六个数据集上超越基于NLI的模型,且不依赖微调或特定数据集训练。
  • 适合追求零样本、无训练成本的多标签分类任务的研究者使用。

我们提出dLLM-SetScore,一种无需训练的多标签文本分类方法,利用离散掩码扩散语言模型进行判断。对每个候选标签,生成一个简短的是/否问题,并在单一掩码位置比较两个回答词的概率。该方法不依赖任务微调或文本蕴含数据集训练;仅需200个标注样例的验证集即可确定阈值、温度参数和提示措辞。实验表明,将所有标签放入同一提示会导致强烈的位置不对称:在GoEmotions数据集上首个回答槽位预测为正类的比例达99.4%,在Reuters上为100%。而逐标签评分将每个标签置于相同句法位置,使预测结果与标签顺序无关,避免了这一偏差。我们在六组数据集上评估了LLaDA-8B和Dream-7B,对比NLI模型、自回归大模型、SetFit及监督分类器。在五个共享数据集上,指令微调版本在10次比较中有9次提升宏F1,8次提升微F1(未明确因果)。在本协议下,LLaDA-Instruct在Reuters和ECtHR两项指标中达到最高训练自由得分。本文证明了排列不变性,分析了加权汉明损失下的阈值决策,并推导出召回率与F1的短名单上限。探索性的局部联合集合优化步骤反而降低F1,作为负向结果保留。

原文摘要 · Abstract (English)

We present dLLM-SetScore, a training-free method that uses discrete masked-diffusion language models for multi-label text classification. For each candidate label, it asks a short yes/no question and compares the probabilities of the two answer tokens at one masked position. The method uses no task-specific fine-tuning or training on textual-entailment datasets; a 200-example labelled validation slice selects thresholds, temperature, and prompt wording. We first show that placing all labels in one prompt creates a strong slot-position asymmetry: the first answer slot is predicted positive on $99.4\%$ of GoEmotions examples and $100\%$ of Reuters examples. Per-label scoring places every label in the same syntactic position, making predictions invariant to label order and avoiding this artifact. We evaluate LLaDA-8B and Dream-7B on six datasets against NLI models, an autoregressive LLM, SetFit, and supervised classifiers. On the five datasets shared by both diffusion families, Instruct checkpoints improve macro-F1 in 9 of 10 comparisons and micro-F1 in 8 of 10, although these comparisons do not identify the cause. Within our protocol, LLaDA-Instruct records the highest training-free values for both Reuters and ECtHR metrics. We prove permutation invariance, characterize thresholded decisions under weighted Hamming loss, and derive shortlist ceilings for recall and F1. An exploratory local Joint Set Refinement step lowers F1 from biased and unbiased initializations and is retained as a negative result.

多标签分类扩散模型零样本无训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。