用开源模型实现接近闭源模型的表格问答能力
Team Anotheroption at SemEval-2025 Task 8: Bridging the Gap Between Open-Source and Proprietary LLMs in Table QA
- 融合文本转SQL/代码、自校正与检索增强生成的全流程系统
- 在竞赛中达80%准确率，性能媲美闭源大模型
- 适合关注开源模型应用落地的研究者与开发者
本文介绍了为 SemEval 2025 Task 8：表格数据上的问答任务所开发的系统。该方法整合了文本转SQL与文本转代码生成模块、自校正机制以及检索增强生成（RAG），并由大语言模型统一调度，形成端到端（E2E）流程。通过消融实验分析各模块影响，揭示当前领域的挑战。在竞赛评估中，系统取得80%的准确率，在38支参赛队伍中排名前13。结果表明，该流程显著提升了开源模型在表格问答任务中的表现，达到与闭源大模型相当的水平。代码已开源。
原文摘要 · Abstract (English)
This paper presents a system developed for SemEval 2025 Task 8: Question Answering (QA) over tabular data. Our approach integrates several key components: text-to-SQL and text-to-code generation modules, a self-correction mechanism, and a retrieval-augmented generation (RAG). Additionally, it includes an end-to-end (E2E) module, all orchestrated by a large language model (LLM). Through ablation studies, we analyzed the effects of different parts of our pipeline and identified the challenges that are still present in this field. During the evaluation phase of the competition, our solution achieved an accuracy of 80%, resulting in a top-13 ranking among the 38 participating teams. Our pipeline demonstrates a significant improvement in accuracy for open-source models and achieves a performance comparable to proprietary LLMs in QA tasks over tables. The code is available at GitHub repository.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。