用检索代替训练,让法律标签自动标注更高效且不乱编。
Retrieval-Based Multi-Label Legal Annotation: Extensible, Data-Efficient and Hallucination-Free

- 用冻结的检索模型将文档和标签映射到向量空间,通过最近邻匹配预测标签。
- 在100个标签的Eurlex数据集上,比GPT-5.2零样本提升宏平均F1至49.12,计算量减少20-30倍。
- 仅需100个样本即可接近全监督效果,且杜绝生成模型的标签幻觉问题。
多标签法律标注需为长篇事实密集型文书从大规模、动态演化的分类体系中分配多个标签,常面临标注数据有限的问题。传统参数化编码器需针对任务重新训练,标签集变动时须重复训练;而提示大模型生成则成本高昂且标签空间扩大时性能下降。本文将法律标注建模为检索任务:使用冻结的检索模型对文档与标签描述进行嵌入,通过嵌入空间中的k近邻预测标签,实现通过重新嵌入与重索引更新模型,无需梯度反向传播。在三个法律数据集(ECtHR-A、ECtHR-B、Eurlex,含100个标签)上,检索方法达到竞争性准确率并展现强数据效率;在Eurlex上,Qwen-8B检索将宏平均F1从GPT-5.2零样本的40.41提升至49.12,计算量降低20-30倍。仅用100个训练样本,检索在ECtHR-A上微平均F1达48.29,近乎是分层Legal-BERT的两倍(27.87)。我们还量化了生成推理的可靠性缺陷:在确定性解码下,GPT-5.2在0.12%-0.9%的测试样本中生成了不在给定分类体系内的标签。相比之下,检索方法严格遵守预定义标签集,从根本上杜绝幻觉。结果表明,基于检索模型的标注器是高基数、快速变化法律标签空间的实用且可部署的替代方案。
原文摘要 · Abstract (English)
Multi-label legal annotation requires assigning multiple labels from large, evolving taxonomies to long, fact-intensive documents, often under limited supervision. Parametric encoders typically require task-specific training and retraining when the label set changes, while prompting generative large language models becomes costly and degrades as the label space grows. We cast legal annotation as retrieval: we embed documents and label descriptions with a frozen retrieval model and predict labels via k-nearest neighbors in the embedding space, enabling updates by re-embedding and re-indexing rather than gradient-based backpropagation. Across three legal datasets (ECtHR-A, ECtHR-B, and Eurlex with 100 labels), retrieval achieves competitive accuracy and strong data efficiency; on Eurlex, Qwen-8B retrieval improves Macro-F1 from 40.41 (GPT-5.2, zero-shot) to 49.12 while reducing estimated compute by 20-30 times compared to fine-tuning. With only (N=100) training samples, retrieval nearly doubles Micro-F1 over hierarchical Legal-BERT on ECtHR-A (48.29 vs. 27.87). We also quantify a reliability failure mode of generative inference: GPT-5.2 hallucinates labels outside the provided taxonomy in 0.12-0.9% of test samples under deterministic decoding. In contrast, retrieval strictly respects defined label sets, eliminating hallucination by design. These results suggest retrieval-model-based annotators are a practical, deployable alternative for high-cardinality and rapidly changing legal label spaces.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。