用多个大模型合起来做分类，准确率比单个模型高65%。
Majority Rules: LLM Ensemble is a Winning Approach for Content Categorization
- 多个大模型投票决定分类结果，提升决策稳定性。
- 在8660条数据上，最高提升65%的F1分数。
- 适合需要高精度、少人工标注的分类任务。
本研究提出一种基于大语言模型（LLM）的集成框架eLLM，用于非结构化文本分类。通过整合十个最先进的LLM，在相同的零样本条件下，使用由8,660个样本组成的真人标注语料库和IAB层级分类体系进行评估。个体模型因将语义丰富的文本压缩为稀疏类别表示而性能趋于瓶颈，而eLLM通过数学化的集体决策模型与合理的聚合规则，显著提升鲁棒性与准确率。实验显示，与最强单模型相比，eLLM在F1-score上最高提升65%。该方法接近人类专家水平，可大幅减少对人工标注的依赖，提供一种可扩展、可靠的分类解决方案。
原文摘要 · Abstract (English)
This study introduces an ensemble framework for unstructured text categorization using large language models (LLMs). By integrating multiple models, the ensemble large language model (eLLM) framework addresses common weaknesses of individual systems, including inconsistency, hallucination, category inflation, and misclassification. The eLLM approach yields a substantial performance improvement of up to 65\% in F1-score over the strongest single model. We formalize the ensemble process through a mathematical model of collective decision-making and establish principled aggregation criteria. Using the Interactive Advertising Bureau (IAB) hierarchical taxonomy, we evaluate ten state-of-the-art LLMs under identical zero-shot conditions on a human-annotated corpus of 8{,}660 samples. Results show that individual models plateau in performance due to the compression of semantically rich text into sparse categorical representations, while eLLM improves both robustness and accuracy. With a diverse consortium of models, eLLM achieves near human-expert-level performance, offering a scalable and reliable solution for taxonomy-based classification that may significantly reduce dependence on human expert labeling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。