提出uBERT模型，加速处理超长法律文本
No Argument Left Behind: Overlapping Chunks for Faster Processing of Arbitrarily Long Legal Texts
- 融合Transformer与RNN结构，用重叠分块处理任意长度文本
- 在长文本上比BERT+LSTM快，且比ULMFiT提升显著
- 适合需要高效处理海量法律文书的机构或研究者
在巴西司法系统（全球规模最大的司法体系）因数百万案件积压而面临危机的背景下，开发高效的法律文本分析方法变得尤为迫切。我们提出uBERT，一种结合Transformer与循环神经网络架构的混合模型，可有效处理任意长度的法律文本，同时保持合理的计算开销。实验表明，使用重叠输入时，uBERT在性能上优于BERT+LSTM，并显著快于ULMFiT，尤其适用于长篇法律文档的处理。
原文摘要 · Abstract (English)
In a context where the Brazilian judiciary system, the largest in the world, faces a crisis due to the slow processing of millions of cases, it becomes imperative to develop efficient methods for analyzing legal texts. We introduce uBERT, a hybrid model that combines Transformer and Recurrent Neural Network architectures to effectively handle long legal texts. Our approach processes the full text regardless of its length while maintaining reasonable computational overhead. Our experiments demonstrate that uBERT achieves superior performance compared to BERT+LSTM when overlapping input is used and is significantly faster than ULMFiT for processing long legal documents.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。