通过动态精简标签空间，提升大模型零样本分类准确率。
From Haystack to Needle: Label Space Reduction for Zero-shot Classification
- 迭代筛选候选标签，聚焦最相关类别
- 在7个基准上平均提升宏F1 7.0%（最高14.2%）
- 适合作为零样本分类的高效增强方案
我们提出标签空间缩减（LSR），一种提升大语言模型零样本分类性能的新方法。LSR通过系统性地排序和缩减候选类别，迭代优化分类标签空间，使模型能聚焦于最相关的选项。利用无标签数据与数据驱动模型的统计学习能力，LSR在测试时动态优化标签空间表示。在七个基准上的实验表明，相较于标准零样本分类基线，使用Llama-3.1-70B时，LSR平均提升宏F1 7.0%（最高达14.2%），使用Claude-3.5-Sonnet时平均提升3.3%（最高达11.1%）。为降低LSR的计算开销（每轮需额外一次LLM调用），我们提出将模型蒸馏为概率分类器，实现高效推理。
原文摘要 · Abstract (English)
We present Label Space Reduction (LSR), a novel method for improving zero-shot classification performance of Large Language Models (LLMs). LSR iteratively refines the classification label space by systematically ranking and reducing candidate classes, enabling the model to concentrate on the most relevant options. By leveraging unlabeled data with the statistical learning capabilities of data-driven models, LSR dynamically optimizes the label space representation at test time. Our experiments across seven benchmarks demonstrate that LSR improves macro-F1 scores by an average of 7.0% (up to 14.2%) with Llama-3.1-70B and 3.3% (up to 11.1%) with Claude-3.5-Sonnet compared to standard zero-shot classification baselines. To reduce the computational overhead of LSR, which requires an additional LLM call at each iteration, we propose distilling the model into a probabilistic classifier, allowing for efficient inference.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。