轻量级模型GLiClass高效实现零样本序列分类,兼顾准确率与速度。
GLiClass: Generalist Lightweight Model for Sequence Classification Tasks
- 基于GLiNER架构改进,支持零/少样本学习
- 在多标签任务中达到嵌入式方法的效率水平
- 适合需要快速适应新分类需求的场景
分类是人工智能应用中最常见的任务之一,常用于数据过滤、排序和分组。现代AI系统需处理海量输入数据,早期流程中的错误会向下传递,因此高效率与高准确率至关重要。同时,分类需求可能随用户变化而动态调整,要求模型具备强零样本能力。尽管生成式大模型因通用性成为零样本分类主流,但存在指令遵循不一致和计算低效的问题。交叉编码器虽常用于RAG重排序,却因必须逐对处理文本-标签而效率低下,尤其在标签集较大时更明显。嵌入式方法虽效率高,但在涉及逻辑与语义约束的复杂场景下表现不佳。本文提出GLiClass,将GLiNER架构适配于序列分类任务,实现与嵌入式方法相当的效率与精度,同时保留零样本与少样本学习的灵活性。此外,我们采用近端策略优化(PPO)改进多标签文本分类,可在数据稀疏或依赖人类反馈的条件下训练分类器。
原文摘要 · Abstract (English)
Classification is one of the most widespread tasks in AI applications, serving often as the first step in filtering, sorting, and categorizing data. Since modern AI systems must handle large volumes of input data and early pipeline stages can propagate errors downstream, achieving high efficiency and accuracy is critical. Moreover, classification requirements can change dynamically based on user needs, necessitating models with strong zero-shot capabilities. While generative LLMs have become mainstream for zero-shot classification due to their versatility, they suffer from inconsistent instruction following and computational inefficiency. Cross-encoders, commonly used as rerankers in RAG pipelines, face a different bottleneck: they must process text-label pairs sequentially, significantly reducing efficiency with large label sets. Embedding-based approaches offer good efficiency but struggle with complex scenarios involving logical and semantic constraints. We propose GLiClass, a novel method that adapts the GLiNER architecture for sequence classification tasks. Our approach achieves strong accuracy and efficiency comparable to embedding-based methods, while maintaining the flexibility needed for zero-shot and few-shot learning scenarios. Additionally, we adapted proximal policy optimization (PPO) for multi-label text classification, enabling training classifiers in data-sparse conditions or from human feedback.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。