用智能调度和内存自适应,让大模型标注更高效稳定。
A Scalable Pipeline for LLM-Teacher Distillation Labeling: Work-Stealing Job Scheduling and Memory-Aware GPU Concurrency
- 工作窃取队列+原子抢任务,故障时仍能保证任务不丢失。
- 在高负载下吞吐量是静态分片的3.4倍,一半节点崩溃仍0任务丢失。
- 适配不同显卡大小,可复现于单机,适合大规模文本标注场景。
用大模型教师标注大规模文本数据已成为训练数据规模化的重要路径。面对数百万条数据,手动分批标注不可行,核心问题在于:每美元能买到的标签质量如何,以及如何在负载不均、易出错的情况下保持GPU集群持续高效运转。本文提出一个简单可复现的流水线:首先,采用工作窃取环形队列机制,每个工作节点先处理自有队列,再从后继节点窃取任务,通过原子比较并交换实现唯一任务申领,故障后通过过期申领清理保障容错;该协议仅需存储层支持比较并交换,作者基于单个SQLite文件实现,无需依赖,可在单机上完全复现实验。其次,设计内存感知的并发规则,根据显卡容量动态调整每节点并行模型数,确保代码在不同设备上安全运行。第三,提出重标注评估方法,让教师重新标注已有真实标签的数据集,标签质量以一致率衡量,成本由实测吞吐决定。在负载不均情况下,该方案吞吐量达静态分片的3.4倍,零偏差时性能相当;半数节点中途崩溃时,任务零丢失(静态分片损失953条);在讽刺与情感任务上,对指令微调教师进行质量与成本测量。所有实验基于公开数据与通用硬件完成,代码、测试与运行日志均已开源。
原文摘要 · Abstract (English)
Labeling large text corpora with LLM teachers has become a practical route to training data at scale. At millions of items, hand-labeling every batch is not feasible, and two questions dominate: what label quality a teacher buys per dollar, and how to keep a fleet of GPU workers busy under skewed, failure-prone workloads. We present a simple, reproducible pipeline that addresses both. First, a work-stealing ring pool: each worker owns a queue, drains it first, and then steals from ring successors, with exactly-once task claims via atomic conditional writes and crash tolerance via stale-claim sweeping. The claim protocol requires only a compare-and-set primitive from its storage layer; we implement it on a single SQLite file, which makes the reference implementation dependency-free and the experiments reproducible on one machine. Second, a memory-aware concurrency rule that sizes per-node parallelism by how many model copies fit on the GPU, so the same code runs safely across device sizes. Third, a relabeling benchmark methodology in which the teacher relabels a public dataset that already has gold labels, so quality reduces to an agreement measurement and cost follows from measured throughput. Under skewed load the pool sustains up to 3.4 times the throughput of static sharding while matching it at zero skew, loses 0 of 2,000 tasks when half the workers are killed mid-run (static sharding loses 953), and yields measured quality and cost points for an instruction-tuned teacher on irony and sentiment tasks. All experiments run on public data and commodity hardware; code, tests, and run logs are released.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。