构建巴西个税法问答数据集，支持带引用的法律智能问答。
BR-TaxQA-R: A Dataset for Question Answering with References for Brazilian Personal Income Tax Law, including case law
- 基于巴西税务机构官方问答，融合法规与判例构建数据集。
- 自研RAG系统在相关性上优于ChatGPT等商用工具。
- 强调专家审核对高风险领域AI答案合法性至关重要。
本文提出BR-TaxQA-R，一个面向巴西个人所得税法的问答带引用数据集。该数据集包含巴西税务局2024年发布的715个官方问答，并整合了联邦行政复审理事会（CARF）的法规和行政裁决。我们采用OpenAI嵌入进行检索，GPT-4o-mini生成答案，构建了增强型生成（RAG）流程。对比不同文本分割策略，并使用RAGAS指标评估系统性能，结果表明自研RAG在响应相关性上优于ChatGPT和Perplexity.ai，而商用模型在事实正确性和流畅性上表现更佳。研究揭示了法律严谨性与语言自然性之间的权衡。关键结论是：在税收等高风险领域，人工专家评审仍不可或缺。数据集已公开于https://huggingface.co/datasets/unicamp-dl/BR-TaxQA-R。
原文摘要 · Abstract (English)
This paper presents BR-TaxQA-R, a novel dataset designed to support question answering with references in the context of Brazilian personal income tax law. The dataset contains 715 questions from the 2024 official Q\&A document published by Brazil's Internal Revenue Service, enriched with statutory norms and administrative rulings from the Conselho Administrativo de Recursos Fiscais (CARF). We implement a Retrieval-Augmented Generation (RAG) pipeline using OpenAI embeddings for searching and GPT-4o-mini for answer generation. We compare different text segmentation strategies and benchmark our system against commercial tools such as ChatGPT and Perplexity.ai using RAGAS-based metrics. Results show that our custom RAG pipeline outperforms commercial systems in Response Relevancy, indicating stronger alignment with user queries, while commercial models achieve higher scores in Factual Correctness and fluency. These findings highlight a trade-off between legally grounded generation and linguistic fluency. Crucially, we argue that human expert evaluation remains essential to ensure the legal validity of AI-generated answers in high-stakes domains such as taxation. BR-TaxQA-R is publicly available at https://huggingface.co/datasets/unicamp-dl/BR-TaxQA-R.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。