用代码生成模型自动构建弱监督标签源，提升覆盖率同时保持准确率。
ScriptoriumWS: A Code Generation Assistant for Weak Supervision
- 用提示工程驱动代码生成模型自动生成弱监督规则
- 相比人工编写的规则，覆盖范围提升但准确率相当
- 适合需要快速构建标注规则的领域专家或数据工程师
弱监督是一种克服标注数据瓶颈的流行框架：通过多个噪声但廉价的数据源提供标签猜测，并聚合生成高质量伪标签。这些数据源通常由领域专家编写的小程序构成——成本较高。本文提出使用代码生成模型作为编程助手，自动创建弱监督规则。我们研究了多种提示策略，最终采用包含多层信息的多层次提示方法。同时探索了人工与生成规则的最佳融合方式。基于此，我们构建了 ScriptoriumWS 系统，相比手工编写的规则，在保持相同准确率的同时显著提升覆盖范围。
原文摘要 · Abstract (English)
Weak supervision is a popular framework for overcoming the labeled data bottleneck: the need to obtain labels for training data. In weak supervision, multiple noisy-but-cheap sources are used to provide guesses of the label and are aggregated to produce high-quality pseudolabels. These sources are often expressed as small programs written by domain experts -- and so are expensive to obtain. Instead, we argue for using code-generation models to act as coding assistants for crafting weak supervision sources. We study prompting strategies to maximize the quality of the generated sources, settling on a multi-tier strategy that incorporates multiple types of information. We explore how to best combine hand-written and generated sources. Using these insights, we introduce ScriptoriumWS, a weak supervision system that, when compared to hand-crafted sources, maintains accuracy and greatly improves coverage.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。