用大规模搜索优化标签,但算力再快也离不开人类初始标注。
Reverse Supervision at Scale: Exponential Search Meets the Economics of Annotation
- 反向监督:在无标签数据中搜索最优标签,最小化小样本有标签集的误差。
- 搜索空间为2^n,即使超算加速,复杂度仍指数级,无法绕过监督需求。
- 生成模型可作标签放大器,但需人类设定目标和校准,确保质量与任务对齐。
我们分析了一种反向监督策略,即在大规模无标签数据集 $B$ 中搜索标签组合,以最小化小规模有标签数据集 $A$ 上的误差。搜索空间大小为 $2^n$,即使采用量子计算或大规模并行硬件实现常数倍加速,其复杂度仍保持指数级。因此,任意快速但非指数级加速的计算无法消除对信息性标签或先验知识的需求。实践中,机器学习流程仍需人类初始投入:定义任务目标、划分类别,并提供具有代表性的种子标注集,以注入归纳偏置并使模型对齐任务语义。若生成式AI产生的合成标签达到人类水准,并由人类指定的目标、种子监督和验证锚定,则可部分替代人工标注。在此视角下,生成模型充当‘标签放大器’,通过主动学习、半监督及自训练循环,利用少量人工标注的核心数据扩展标签规模;而人类则负责校准、漂移检测与失败审计。因此,极致算力仅缩短实际运行时间,不改变学习的根本监督需求;初始人类(或人类级)输入仍是系统对接目标任务的关键基础。
原文摘要 · Abstract (English)
We analyze a reversed-supervision strategy that searches over labelings of a large unlabeled set \(B\) to minimize error on a small labeled set \(A\). The search space is \(2^n\), and the resulting complexity remains exponential even under large constant-factor speedups (e.g., quantum or massively parallel hardware). Consequently, arbitrarily fast -- but not exponentially faster -- computation does not obviate the need for informative labels or priors. In practice, the machine learning pipeline still requires an initial human contribution: specifying the objective, defining classes, and providing a seed set of representative annotations that inject inductive bias and align models with task semantics. Synthetic labels from generative AI can partially substitute provided their quality is human-grade and anchored by a human-specified objective, seed supervision, and validation. In this view, generative models function as \emph{label amplifiers}, leveraging small human-curated cores via active, semi-supervised, and self-training loops, while humans retain oversight for calibration, drift detection, and failure auditing. Thus, extreme computational speed reduces wall-clock time but not the fundamental supervision needs of learning; initial human (or human-grade) input remains necessary to ground the system in the intended task.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。