arXiv:2606.02971cs.CL2026-06

构建EU法规报告义务提取数据集,验证领域预训练与高效微调的价值

EURO-5K: When Does Domain Pretraining Matter? Benchmarking Transformers for EU Reporting Obligation Extraction

论文配图:EURO-5K: When Does Domain Pretraining Matter? Benchmarking Transformers for EU Reporting Obligation Extraction
图 1 · 摘自论文原文
  • 构建5000+句级标注的EU法规提取数据集,含复杂负例
  • 法律领域预训练在小样本下加速学习,参数高效微调更优
  • 生成式与判别式模型均有效,适合监管合规自动化场景

从欧盟立法中提取报告义务对评估和降低监管负担至关重要。但区分报告要求与结构相似条款需专业法律理解。现有法律NLP方法缺乏专用数据集与明确评估标准。我们构建了EURO-5K,涵盖136部欧盟立法中的5000+句级报告义务及挑战性负例。在此数据集上,我们训练并比较了判别式(BERT类)与生成式(LLM类)跨度提取模型,评估全量微调与参数高效QLoRA,并对比基线方法(模式匹配、依赖解析、少样本提示)。结果表明:全量微调的通用与法律BERT性能相当(F1=0.89);微调的LLM在句子级提取上达到编码器水平;法律预训练对生成模型提升有限;但在适应能力受限时,法律BERT的参数高效微调显著优于通用版本。学习曲线分析显示法律预训练可加速小样本下的早期学习。所有方法在约3000样本后趋于收敛,表明数据集充足。跨数据集评估在两个外部监管语料上验证了模型的专门化提取能力而非泛化分类。我们发布EURO-5K、训练模型及交互式演示(含可解释可视化与结构化RDF导出),证明两种范式与参数高效训练均为监管合规自动化的实用工具。

原文摘要 · Abstract (English)

Extracting reporting obligations from EU legislation is critical for assessing and reducing regulatory reporting burden. However, distinguishing reporting requirements from structurally similar provisions requires specialised legal understanding. Current legal NLP methods lack specialised datasets with clear guidelines and comparative evaluation of extraction paradigms and domain adaptation strategies. We curate EURO-5K, a corpus of sentence-level reporting obligations and challenging negative examples from 136 EU legislative acts. On this dataset, we train and compare discriminative token-classification models (BERT-style) and generative span-extraction models (LLMs), evaluating both full fine-tuning and parameter-efficient QLoRA against baselines (pattern and dependency-based extraction, few-shot prompting). Results show that fully fine-tuned generic and legal BERT models achieve similar performance (0.89 F1), while fine-tuned LLMs match encoder accuracy for sentence-level extraction. Legal pretraining offers only small gains for generative models. In contrast, it is clearly beneficial when adaptation capacity is constrained, as parameter-efficient tuning of Legal-BERT outperforms its generic counterpart. Learning curve analysis demonstrates that legal pretraining accelerates early learning with minimal data. All approaches converge around 3K samples with diminishing returns thereafter, validating dataset sufficiency. Cross-dataset evaluation on two external regulatory corpora shows that our models behave as specialised reporting obligation extractors rather than generic regulatory classifiers. We release EURO-5K, trained models, and an interactive demo with explainability visualizations and structured RDF export. These demonstrate that both paradigms and parameter-efficient training provide practical tools for regulatory compliance automation.

法律NLP信息抽取参数高效微调监管合规

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。