arXiv:2606.08090cs.DBcs.AI2026-06

用自适应两阶段方法提升大模型语义过滤速度,兼顾精度与效率。

Fast LLM-Based Semantic Filtering: From a Unified Framework to an Adaptive Two-Phase Method

论文配图:Fast LLM-Based Semantic Filtering: From a Unified Framework to an Adaptive Two-Phase Method
图 1 · 摘自论文原文
  • 融合聚类与在线训练代理,按需调用大模型,减少无效计算。
  • 用文档置信度作软标签训练代理,提升边界样本识别能力。
  • 动态校准安全边际,避免盲目增加调用次数,适合高并发数据处理场景。

在文档语料上以90%准确率评估自然语言是/否谓词——即语义过滤——是大模型数据处理的核心任务。直接对每篇文档调用大模型(原生接口)成本过高,因此采用级联结构,用快速代理替代部分调用。当前方法存在四大局限:(1) 各类级联方法(无模型聚类、预构建小模型代理、在线训练代理)固定使用单一表示与流程,仅适用于特定查询范围;(2) 最强的在线代理基于密集嵌入的双编码器,缺乏对细粒度标记证据的支持;(3) 代理仅使用二值标签训练,浪费了原生模型在边界文档上的置信度信息;(4) 现有校准方式添加统一安全余量,混淆真实不确定性与小样本噪声,导致级联开销增加。本文提出四点改进:(1) 自适应组合不同家族方法——先无模型聚类,必要时启用在线代理,且原生调用共享于各阶段;(2) 以现成的标记感知模型替代余弦双编码器;(3) 使用原生模型的每文档置信度作为软标签训练代理;(4) 仅在标注样本稀疏区域添加安全余量。首次将原生模型置信度用于三个目的:查询难度指示器、任何代理级联的最小原生调用下界、代理训练的软标签。在三个10,000文档语料上,90%准确率目标下,方法比最优前序方法快1.6–2.0倍,95%查询达标;基于伯努利误差率(BER)的下界表明未来仍有约4–20倍优化空间。

原文摘要 · Abstract (English)

Evaluating a natural-language yes/no predicate over a document corpus under an accuracy target - the semantic filter - is a cornerstone of LLM-based data processing. Calling the LLM on every document (the oracle) is prohibitive, so cascades pair the oracle with a fast proxy. As deployed today, they leave four limitations on the table. (1) Each cascade family - model-free clustering, prebuilt small-LLM proxies, online-trained proxies - commits to a single representation and pipeline, and wins on only a narrow query regime. (2) The strongest online proxy invests in a custom training scheme on a bi-encoder over dense embeddings, missing the token-level evidence richer predicates require. (3) The proxy is trained against binary yes/no labels, wasting the LLM's per-document confidence at the boundary documents it most needs to learn. (4) Existing calibrations add a uniform safety margin, conflating genuine proxy uncertainty with small-sample noise and inflating cascade cost. We address these by (1) composing families adaptively - model-free clustering first, online proxy only when needed, with oracle calls shared across phases; (2) replacing the cosine bi-encoder with a hybrid of off-the-shelf token-aware models; (3) training the proxy with the oracle's per-document confidence as a soft label; and (4) a calibration that adds the safety margin only where the labeled sample is sparse. We are also the first to use the oracle's per-document confidence for three purposes: a query-level difficulty compass, a lower bound on the minimum oracle calls any proxy-based cascade can make, and the proxy's soft training label. At a 90% accuracy target on three 10K-document corpora, our methods are 1.6-2.0x faster than the best prior method per corpus and meet the target on 95% of queries; the BER-derived lower bound indicates a further ~4-20x of headroom for future work.

大模型语义过滤高效推理自适应系统

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。