arXiv:2607.18825cs.CLcs.AI2026-07中稿 · AI and Law Journal被引 1

构建面向印度法律的AI问答系统,验证其在真实法律场景中的有效性。

AILQA: Evaluating AI-Driven Legal Question Answering Systems for the Indian Legal System

论文配图:AILQA: Evaluating AI-Driven Legal Question Answering Systems for the Indian Legal System
图 1 · 摘自论文原文
  • 采用检索增强生成技术融合多种大模型,提升法律问答准确性。
  • 在全印律师资格考试测试中,部分AI答案评分高于参考答案。
  • 适合法律科技研究者及法律AI系统开发者参考使用。

本研究提出针对印度法律体系的AI驱动法律问答系统AILQA,利用多种嵌入与生成模型,包括最新大型语言模型(LLMs),应对印度法律文本复杂多样的挑战,提升法律问题回答的准确性和可靠性。通过词法与语义指标结合专家法律反馈进行严格评估,结果表明检索增强生成(RAG)范式在复杂法律领域显著提升回答质量。此外,系统在全印律师资格考试(AIBE)标准测试中表现优异,部分AI生成回答在评分中超越现有参考答案,尤其在提供准确且相关支持细节方面表现突出。该发现基于特定数据集和评分标准,不代表模型普遍优于专业法律人士。研究还指出精准上下文依赖与模型幻觉风险等挑战,并为未来法律AI研究提出改进方向。旨在推动更高效、可及的法律决策支持系统发展。

原文摘要 · Abstract (English)

This comprehensive study introduces an advanced Artificial Intelligence for Indian Legal Question Answering (AILQA) system tailored to the Indian legal context. AILQA leverages a variety of embedding and generative models, including recent Large Language Models (LLMs), to address the unique challenges posed by the intricate and diverse nature of Indian legal texts and to enhance the accuracy and reliability of responses to legal questions. We conducted rigorous evaluations using both lexical and semantic metrics, enriched by expert legal feedback, to ensure relevance and accuracy. Our findings underscore the effectiveness of the Retrieval-Augmented Generation (RAG) paradigm in improving answer quality, particularly in complex legal domains. Additionally, we assessed performance on standardized tests such as the All India Bar Examination (AIBE), thereby providing a robust benchmark for practical applications. Under the study's evaluation protocol, some AI-generated responses received higher ratings than the available reference answers, particularly when they contained accurate and relevant supporting details. This finding is specific to the evaluated dataset and rating criteria and should not be interpreted as evidence that the models generally outperform qualified legal professionals. We also discuss the challenges encountered, such as the need for precise context and the risks of model hallucination, and propose directions for future research to further refine AI capabilities in the legal field. This study aims to pave the way for enhanced legal decision-support systems, making them more accessible and effective for legal professionals and the public alike.

法律AI问答系统RAG

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。