用大模型生成Python代码实现零样本表格问答，效果优于传统方法。
ITUNLP at SemEval-2025 Task 8: Question-Answering over Tabular Data: A Zero-Shot Approach using LLM-Driven Code Generation
- 通过优化提示词让大模型生成可执行的Pandas代码
- 在两个子任务中分别取得第八和第六名，优于基线
- 适合需要快速适配新领域表格数据的应用场景
本文介绍了我们在SemEval-2025 Task 8：DataBench（表格数据上的问答）中的系统。该任务包含两个子任务：DataBench QA（子任务I）和DataBench Lite QA（子任务II），目标是在不同领域的给定表格数据上进行问答。为应对两个子任务，我们提出了一种零样本解决方案，重点利用大语言模型（LLM）驱动的代码生成技术。具体而言，我们构建了一个基于开源大模型的Python代码生成框架，通过优化的提示策略生成可执行的Pandas代码。实验表明，不同大模型在代码生成能力上表现各异，且基于代码生成的方法在表格问答任务中显著优于其他替代方案。尽管提交时零样本系统排名未知，但在开源模型类别中，我们的系统在子任务I中位列第八，在子任务II中位列第六，30个参赛系统中超过基线水平。
原文摘要 · Abstract (English)
This paper presents our system for SemEval-2025 Task 8: DataBench, Question-Answering over Tabular Data. The primary objective of this task is to perform question answering on given tabular datasets from diverse domains under two subtasks: DataBench QA (Subtask I) and DataBench Lite QA (Subtask II). To tackle both subtasks, we developed a zero-shot solution with a particular emphasis on leveraging Large Language Model (LLM)-based code generation. Specifically, we propose a Python code generation framework utilizing state-of-the-art open-source LLMs to generate executable Pandas code via optimized prompting strategies. Our experiments reveal that different LLMs exhibit varying levels of effectiveness in Python code generation. Additionally, results show that Python code generation achieves superior performance in tabular question answering compared to alternative approaches. Although our ranking among zero-shot systems is unknown at the time of this paper's submission, our system achieved eighth place in Subtask I and sixth place in Subtask~II among the 30 systems that outperformed the baseline in the open-source models category.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。