用链式模型组合提升标注精度与效率，降低大模型使用成本。
LLM Chain Ensembles for Scalable and Accurate Data Annotation
- 按不确定度逐级调度不同大模型，让强项模型处理擅长任务。
- 链式组合比单个最优模型性能更高，节省大量计算成本。
- 适合需要高质量大规模标注的快速变化领域使用。
大语言模型（LLM）具备零样本分类能力，使其成为快速演化领域中数据标注的可行方案，尤其在高质量标注数据稀缺且获取成本高昂时。然而，大模型的大规模部署可能代价过高。本文提出一种LLM链式集成方法，将多个大模型按顺序排列，根据分类不确定性将数据子集路由至后续模型。该方法发挥各模型优势，在其高置信度区间处理任务，复杂案例则交由更鲁棒的模型处理。实验表明，链式集成方法常优于链中表现最佳的单一模型，同时实现显著的成本节约，使LLM链式集成成为大规模数据标注问题的实际高效解决方案。
原文摘要 · Abstract (English)
The ability of large language models (LLMs) to perform zero-shot classification makes them viable solutions for data annotation in rapidly evolving domains where quality labeled data is often scarce and costly to obtain. However, the large-scale deployment of LLMs can be prohibitively expensive. This paper introduces an LLM chain ensemble methodology that aligns multiple LLMs in a sequence, routing data subsets to subsequent models based on classification uncertainty. This approach leverages the strengths of individual LLMs within a broader system, allowing each model to handle data points where it exhibits the highest confidence, while forwarding more complex cases to potentially more robust models. Our results show that the chain ensemble method often exceeds the performance of the best individual model in the chain and achieves substantial cost savings, making LLM chain ensembles a practical and efficient solution for large-scale data annotation challenges.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。