arXiv:2602.06370cs.CL2026-02被引 1

对比微调模型与大模型提示,发现前者在文本分类中更省成本且更快。

Cost-Aware Model Selection for Text Classification: Multi-Objective Trade-offs Between Fine-Tuned Encoders and LLM Prompting in Production

  • 比较微调编码器和零/少样本大模型提示的文本分类效果
  • 微调模型性能相当甚至更好,成本和延迟低1-2个数量级
  • 适合追求高效稳定的生产系统,不推荐盲目使用大模型

大型语言模型(如GPT-4o和Claude Sonnet 4.5)在开放域推理和生成任务中表现优异,广泛应用于各类NLP场景。然而,对于标签空间固定的结构化文本分类任务,模型选择常仅关注预测性能,忽视生产环境中的实际约束。本文系统比较了两种范式:基于提示的零/少样本大模型与全微调的编码器架构。在IMDB、SST-2、AG News和DBPedia四个基准上评估了预测质量(宏F1)、推理延迟和货币成本。将模型评估建模为多目标决策问题,通过帕累托前沿投影和参数化效用函数分析不同部署场景下的权衡。结果表明,BERT家族的微调编码器模型在分类性能上具有竞争力,甚至更优,同时推理成本和延迟比零/少样本大模型低一到两个数量级。研究建议:对标准文本分类任务过度依赖大模型会导致系统级效率下降。微调编码器应作为结构化NLP流水线的可靠高效组件,而大模型更适合在混合架构中作为补充模块。代码、数据集和评估协议均已公开,以支持可复现性和成本感知的NLP系统设计。

原文摘要 · Abstract (English)

Large language models (LLMs) such as GPT-4o and Claude Sonnet 4.5 have demonstrated strong capabilities in open-ended reasoning and generative language tasks, leading to their widespread adoption across a broad range of NLP applications. However, for structured text classification problems with fixed label spaces, model selection is often driven by predictive performance alone, overlooking operational constraints encountered in production systems. In this work, we present a systematic comparison of two contrasting paradigms for text classification: zero- and few-shot prompt-based large language models, and fully fine-tuned encoder-only architectures. We evaluate these approaches across four canonical benchmarks (IMDB, SST-2, AG News, and DBPedia), measuring predictive quality (macro F1), inference latency, and monetary cost. We frame model evaluation as a multi-objective decision problem and analyze trade-offs using Pareto frontier projections and a parameterized utility function reflecting different deployment regimes. Our results show that fine-tuned encoder-based models from the BERT family achieve competitive, and often superior, classification performance while operating at one to two orders of magnitude lower cost and latency compared to zero- and few-shot LLM prompting. Overall, our findings suggest that indiscriminate use of large language models for standard text classification workloads can lead to suboptimal system-level outcomes. Instead, fine-tuned encoders emerge as robust and efficient components for structured NLP pipelines, while LLMs are better positioned as complementary elements within hybrid architectures. We release all code, datasets, and evaluation protocols to support reproducibility and cost-aware NLP system design.

文本分类大模型成本优化模型选择

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。