arXiv:2502.18179cs.CLcs.AI2025-02EMNLP被引 10

用LLM提取复杂版面文档信息,调参后效果可媲美专用模型。

Problem Solved? Information Extraction Design Space for Layout-Rich Documents using LLMs

  • 通过分块、提示工程等方法优化输入与模型交互方式。
  • 最佳配置下比通用基线提升13.3至37.5个F1点。
  • 提供开源测试套件,适合希望免微调的开发者使用。

本文定义并探索了利用大语言模型(LLMs)从布局丰富的文档中进行信息抽取(IE)的设计空间。核心挑战包括数据结构化、模型参与和输出优化。研究分析了输入表示、分块策略、提示设计、LLM选择及多模态模型等子问题。通过新发布的开源测试套件LayIE-LLM,在两个数据集上对比传统微调模型,结果表明:经优化的LLM配置相比通用基线配置,性能提升13.3–37.5 F1点。采用一次只变一个因子(OFAT)的方法,仅需2.8%的计算量即可接近全因子实验的最佳结果(仅差0.8–1.8 F1点)。研究表明,若配置得当,通用型LLM可达到专业模型水平,为无需微调的信息抽取提供高效替代方案。测试套件已开源:https://github.com/gayecolakoglu/LayIE-LLM。

原文摘要 · Abstract (English)

This paper defines and explores the design space for information extraction (IE) from layout-rich documents using large language models (LLMs). The three core challenges of layout-aware IE with LLMs are 1) data structuring, 2) model engagement, and 3) output refinement. Our study investigates the sub-problems and methods within these core challenges, such as input representation, chunking, prompting, selection of LLMs, and multimodal models. It examines the effect of different design choices through LayIE-LLM, a new, open-source, layout-aware IE test suite, benchmarking against traditional, fine-tuned IE models. The results on two IE datasets show that LLMs require adjustment of the IE pipeline to achieve competitive performance: the optimized configuration found with LayIE-LLM achieves 13.3--37.5 F1 points more than a general-practice baseline configuration using the same LLM. To find a well-working configuration, we develop a one-factor-at-a-time (OFAT) method that achieves near-optimal results. Our method is only 0.8--1.8 points lower than the best full factorial exploration with a fraction (2.8%) of the required computation. Overall, we demonstrate that, if well-configured, general-purpose LLMs match the performance of specialized models, providing a cost-effective, finetuning-free alternative. Our test-suite is available at https://github.com/gayecolakoglu/LayIE-LLM.

信息抽取LLM应用文档解析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。