梳理129个大模型提示数据集，发现提示文本的语法语义特征可高效用于筛选与路由。
A Survey of LLM Prompt Datasets: Taxonomy, Linguistic Patterns, and Practical Uses
- 构建统一分类体系，分析提示文本的词法、句法和语义模式。
- 基于63维结构特征实现近91%的精准率，延迟降低近一半。
- 适合研究提示工程、模型评估与高效推理系统的设计者。
我们收集了129个公开的大语言模型提示数据集，总规模超过1.22TB，包含67300万条实例，并建立统一分类体系。通过七个数据集的深入分析，识别出提示文本在词汇、句法和语义层面区别于普通文本的特征。这些特征被应用于提示过滤、源域路由和生成响应质量分析。一个63维的结构化语言特征集可在CPU上提取，其性能达到GPU句向量嵌入91%以上的F1值，单请求延迟减少近半。跨数据集路由中，句向量嵌入获得0.74的宏平均F1，结构特征子集达0.61，远超随机水平（0.20）。在控制提示长度后，结构特征仍与生成质量负相关；类型与标记比例的正相关性也显著减弱。结果支持结合高效结构特征与稳健语义路由的提示处理流水线。数据集目录、标注与代码已开源至https://github.com/ymzhang-cs/prompt-dataset-analysis。
原文摘要 · Abstract (English)
We compile 129 public LLM prompt datasets with more than 1.22TB and more than 673M instances and organize them into a unified taxonomy. We use seven datasets for detailed analysis and identify lexical, syntactic, and semantic patterns that distinguish prompts from general text. We evaluate these features in prompt filtering, source domain routing, and elicited response quality analysis. A 63 dimensional linguistic feature set that can be extracted on a CPU achieves over 91% of the F1 of GPU sentence embeddings and reduces latency per request by nearly half. In cross dataset routing, sentence embeddings reach 0.74 Macro-F1 and a structural feature subset reaches 0.61, compared with the chance level of 0.20. Structural routing features remain negatively associated with elicited response quality after prompt length is controlled on UltraFeedback. The apparent positive association of type to token ratio becomes much smaller under the same control. These findings support prompt processing pipelines that combine efficient structural features with robust semantic routing. Our catalogue, annotations, and code are given at https://github.com/ymzhang-cs/prompt-dataset-analysis.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。