批量处理+多变量合并，可省超80%大模型标注成本
Researchers waste 80% of LLM annotation costs by classifying one text at a time
- 将多个文本批量处理并合并多变量至单个提示，大幅减少API调用次数
- 100个样本批量处理时，6款模型准确率仍比单样本基线低不超过2个百分点
- 适合需要高效标注海量文本的社会科学研究者
大语言模型在社会科学文本分类中应用日益广泛，但研究者普遍采用逐条、单变量的提示方式。对10万条文本在4个变量上进行编码需40万次API调用。通过每批25个样本、将最多25个变量合并至单一提示，可将调用次数降至4000次，降低80%以上令牌成本。我们测试了来自四大厂商的8款主流大模型，在3962条专家标注推文上，批量规模从1到1000不等，变量堆叠达25个。六款模型在批量规模达100时，准确率与单样本基线相差不超过2个百分点。最多堆叠10个变量时，结果与单变量编码相当，性能下降主要由任务复杂度决定而非提示长度。在此安全范围内，批量与堆叠带来的测量误差小于真实数据中常见的研究者间分歧。
原文摘要 · Abstract (English)
Large language models (LLMs) are increasingly being used for text classification across the social sciences, yet researchers overwhelmingly classify one text per variable per prompt. Coding 100,000 texts on four variables requires 400,000 API calls. Batching 25 items and stacking all variables into a single prompt reduces this to 4,000 calls, cutting token costs by over 80%. Whether this degrades coding quality is unknown. We tested eight production LLMs from four providers on 3,962 expert-coded tweets across four tasks, varying batch size from 1 to 1,000 items and stacking up to 25 coding dimensions per prompt. Six of eight models maintained accuracy within 2 pp of the single-item baseline through batch sizes of 100. Variable stacking with up to 10 dimensions produced results comparable to single-variable coding, with degradation driven by task complexity rather than prompt length. Within this safe operating range, the measurement error from batching and stacking is smaller than typical inter-coder disagreement in the ground-truth data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。