轻量多语言测试集，秒级运行，可实时检测大模型提示词错误。
Tiny QA Benchmark++: Ultra-Lightweight, Synthetic Multilingual Dataset Generation & Smoke-Tests for Continuous LLM Evaluation
- 用52个英文黄金问题+合成数据生成器，快速构建任意语言/领域测试集。
- 单次运行仅增几秒延迟，却能提前发现提示词、分词器和微调问题。
- 适合开发者在代码提交、提示工程中嵌入自动化质量检查。
Tiny QA Benchmark++（TQB++）提出一种超轻量、多语言的烟雾测试套件，为大语言模型（LLM）流水线提供类似单元测试的安全网数据集，可在数秒内完成，成本极低。基于Comet Opik提示优化SDK开发中的快速反馈需求，TQB++结合一个52项英文黄金测试集（<20 kB）与基于无供应商依赖LiteLLM构建的微型合成数据生成器PyPI包。该生成器允许从业者在任意语言、领域或难度下自动生成小型测试包，同时已提供阿拉伯语、中文、法语、德语、日语、韩语、葡萄牙语、俄语、西班牙语和土耳其语等10种语言的现成数据包。每个数据集均附带Croissant元数据及OpenAI-Evals、LangChain和标准CI工具的即插即用文件，使团队可直接将确定性微基准测试嵌入拉取请求审查、提示工程循环和生产仪表盘，无需占用GPU资源。一次完整TQB++运行仅增加数秒流水线延迟，但能可靠检测提示模板错误、分词器漂移和微调副作用，远早于MMLU或BIG-Bench等大规模基准完成配置。整个框架已开源，旨在加速生成式AI生态中持续、高效的资源节约型质量保障。
原文摘要 · Abstract (English)
Tiny QA Benchmark++ (TQB++) presents an ultra-lightweight, multilingual smoke-test suite designed to give large-language-model (LLM) pipelines a unit-test style safety net dataset that runs in seconds with minimal cost. Born out of the tight feedback-loop demands building the Comet Opik prompt-optimization SDK, where waiting on heavyweight benchmarks breaks developer flow. TQB++ couples a 52-item English gold set (less than 20 kB) with a tiny synthetic-data generator pypi package built on provider-agnostic LiteLLM. The generator lets practitioners mint their own tiny packs in any language, domain, or difficulty, while ten ready-made packs already cover Arabic, Chinese, French, German, Japanese, Korean, Portuguese, Russian, Spanish, and Turkish. Every dataset ships with Croissant metadata and plug-and-play files for OpenAI-Evals, LangChain, and standard CI tools, so teams can drop deterministic micro-benchmarks directly into pull-request gates, prompt-engineering loops, and production dashboards without touching GPU budgets. A complete TQB++ run adds only a few seconds to pipeline latency yet reliably flags prompt-template errors, tokenizer drift, and fine-tuning side-effects long before full-scale suites like MMLU or BIG-Bench would finish configuring. The entire framework is released to accelerate continuous, resource-efficient quality assurance across the generative-AI ecosystem.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。