用批量提示技术降低大模型推荐标注成本，效率提升2.5倍
PromptPack: Scaling LLM Annotation Agents for Online Recommendation
- 将多个广告素材批量处理，共享系统指令减少94%冗余输入
- 批量20时成本降89%，吞吐提升2.5倍，AUC保持不变
- 适合大规模在线推荐中需高效提取结构化特征的场景
在线推荐平台越来越多地使用大语言模型（LLM）从广告创意中提取结构化特征。尽管单次调用的LLM标注代理在实际生产环境中显著提升了点击率（CTR），但按创意逐个提示的模式难以扩展，每次请求中的冗余系统指令占用了94%的计费输入令牌。为突破这一成本瓶颈，我们提出PromptPack，一种可扩展、高吞吐的LLM标注代理。其通过上下文批量处理、共享系统提示、严格的XML结构封装和输出修正层，实现多创意同时处理下的确定性、流水线就绪特征提取。我们采用离线检索基准评估，使用下游逻辑回归排序器。为深入分析代理行为，引入新型指标体积加权绝对提升（VWAL），衡量生成特征的信号质量。相比现有单批生产基线，当批量大小为20时，PromptPack将LLM成本降低89%，吞吐量提升2.5倍，同时完全保留原始AUC性能。
原文摘要 · Abstract (English)
Online recommendation platforms increasingly use Large Language Models (LLMs) to extract structured features from ad creatives. While deploying a single-call LLM annotation agent yields significant Click-Through Rate (CTR) improvements in our live production environment, per-creative prompting is prohibitively expensive to scale. The redundant system instructions sent in every request account for 94% of billed input tokens. To break this cost bottleneck, we introduce PromptPack, a scalable, high-throughput LLM annotation agent. PromptPack achieves this scale via in-context batching, combining a shared system prompt, a strict XML structural envelope, and an output correction layer to ensure deterministic, pipeline-ready feature extraction across multiple creatives simultaneously. We evaluate PromptPack via an offline retrieval benchmark using a downstream logistic-regression ranker. To deeply profile the agent's behavior, we measure AUC and introduce Volume-Weighted Absolute Lift (VWAL), a novel metric capturing the signal quality of the generated features. Compared to our live, unbatched production baseline, PromptPack at batch size 20 cuts our LLM costs by 89% and accelerates throughput by 2.5x while fully preserving AUC.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。