用相似文档动态调整标签阈值,提升多标签分类准确率。
RAPT: Retrieval-Augmented Post-hoc Thresholding for Multi-Label Classification

- 基于检索的后置阈值优化,无需重训练模型
- 工业数据集上宏平均F1达0.87,优于静态阈值两倍
- 适用于文档格式变化大、标签不平衡场景
工业级多标签文档理解系统通过评分和阈值筛选候选标签形成文档标签集,该步骤直接影响下游信息抽取精度与验证成本。实际中,OCR噪声、标签不平衡、实例相关标签数量差异及非对称误判代价导致全局阈值脆弱且难以随文档格式演变维持。本文提出RAPT——一种面向部署的检索增强型后置阈值包装器,可无须重训练直接提升标签集选择效果。RAPT为模型无关设计:任何能提供文档表征用于相似性搜索并输出每标签置信度的预测器(如度量学习编码器或微调Transformer)均可使用。对于每个查询文档,给定分类器的得分向量,RAPT检索相似的文档阈值情形(案例),并基于其结果自适应调整查询文档的标签集选择阈值,通过局部聚合邻居解(如平均标签数、截断校准)确定最终标签集。在工业数据集及六个公开基准上评估显示,结合RAPT的多标签分类器持续优于全局与标签级静态阈值基线。在工业场景中,搭配度量学习模型时,RAPT达到0.87宏平均F1;而微调Transformer变体平均达0.775宏平均F1,较5样本提示的LLM基线性能提升2倍,推理时间减少至少115倍,GPU内存降低13.5倍。
原文摘要 · Abstract (English)
Industrial multi-label document understanding pipelines score candidate labels and threshold or rank them to form a label set per document. This early selection step directly affects the accuracy of downstream information extraction from the document, as well as the associated verification effort. In practice, OCR noise, label imbalance, instance-dependent label cardinality, and asymmetric error costs make global score thresholds brittle and hard to maintain as document formats evolve. We present RAPT, a deployment-oriented retrieval-augmented score thresholding wrapper, applied post-hoc to improve label set selection without retraining the underlying classifier. RAPT is a model-agnostic wrapper: any predictor that provides document representations for similarity search and per label confidence scores can be used, including metric learning encoders and fine-tuned transformer classifiers. For each query document, given a classifier's score vector, RAPT retrieves similar document thresholding situations (cases) and adapts the query's label set selection threshold using their outcomes. The adaptation selects the final label set by locally aggregating neighbour solutions (e.g. average label count, cutoff calibration). Evaluation compared multi-label classifiers (metric learners and transformers) combined with RAPT against global and label-wise thresholding baselines, and against few-shot LLMs. Across an industrial dataset and six public benchmarks, RAPT consistently outperformed global and label-wise static thresholding baselines. In the industrial setting, RAPT achieved its best predictive performance with metric learners, reaching 0.87 Macro-F1, while fine-tuned transformer variants on average achieved 0.775 Macro-F1, outperforming fewshot LLM baselines (K = 5) by 2x and requiring at least 115x less inference time and 13.5x less GPU memory.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。