不用微调,只靠提示词设计,就能高效提取西班牙电费单信息
Information Extraction from Electricity Invoices with General-Purpose Large Language Models
- 用提示词工程替代模型调参,重点优化few-shot和迭代提取策略
- 最佳方案下Gemini F1达97.61%,Mistral-small达96.11%
- 提示词质量远超参数调整,适合企业自动化文档处理场景
从半结构化业务文档中提取信息仍是企业管理系统中的关键挑战。本研究评估通用大语言模型在无需任务特定微调的情况下,从西班牙电费发票中提取结构化信息的能力。基于IDSEM数据集的一个子集,我们在19种参数配置和6种提示策略下,对Gemini 1.5 Pro和Mistral-small两种架构不同的模型进行基准测试。实验框架将提示工程作为主要变量,比较零样本基线与日益复杂的少样本方法及迭代提取策略。结果表明,提示质量远超超参数调优:所有参数配置间的F1分数差异微小,而零样本与最优少样本策略之间的差距超过19个百分点。最佳配置(少样本结合交叉验证)下,Gemini的F1得分为97.61%,Mistral-small为96.11%;文档模板结构成为影响提取难度的主要因素。研究证实,提示设计是提升基于LLM的文档处理精度的关键杠杆,为通用大模型融入企业文档自动化提供了实证框架。
原文摘要 · Abstract (English)
Information extraction from semi-structured business documents remains a critical challenge for enterprise management. This study evaluates the capability of general-purpose Large Language Models to extract structured information from Spanish electricity invoices without task-specific fine-tuning. Using a subset of the IDSEM dataset, we benchmark two architecturally distinct models, Gemini 1.5 Pro and Mistral-small, across 19 parameter configurations and 6 prompting strategies. Our experimental framework treats prompt engineering as the primary experimental variable, comparing zero-shot baselines against increasingly sophisticated few-shot approaches and iterative extraction strategies. Results demonstrate that prompt quality dominates over hyperparameter tuning: the F1-score variation across all parameter configurations is marginal, while the gap between zero-shot and the best few-shot strategy exceeds 19 percentage points. The best configuration (few-shot with cross-validation) achieves an F1-score of 97.61% for Gemini and 96.11% for Mistral-small, with document template structure emerging as the primary determinant of extraction difficulty. These findings establish that prompt design is the critical lever for maximizing extraction fidelity in LLM-based document processing, thereby providing an empirical framework for integrating general-purpose LLMs into business document automation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。