用80亿参数模型实现印度法律辅助,性能超越1750亿参数大模型。
Lightweight Domain Adaptation of a Large Language Model for Legal Assistance in the Indian Context
- 用检索增强生成+提示工程,适配印度本土法律
- 在全印律师考试中达60.08分,优于175B模型的58.72分
- 显著减少幻觉,适合真实法律场景部署
印度公众获取法律援助存在严重缺口,许多人因缺乏法律信息和认知而无法行使权利。本文提出 Legal Assist AI 框架,利用一个80亿参数的量化模型(Llama 3.1)实现高效的印度法律领域适应。该框架结合检索增强生成(RAG)与策略性提示工程,并基于包含超过600份法律文件的高质量语料库,涵盖印度宪法、新颁布的《印度刑法典》(BNS)和《印度公民安全法典》(BNSS)等。在全印律师考试(AIBE)基准测试中,该框架取得60.08%的得分,优于1750亿参数的GPT-3.5 Turbo(58.72%)。同时,系统有效管理并缓解了幻觉问题,满足实际法律应用需求。本文还引入参数效率指数(PEI),证明该80亿模型相较1750亿基线模型具备22倍的参数效率优势,验证了小规模领域适配模型的潜力。
原文摘要 · Abstract (English)
In India, access to legal assistance for the general public has been observed to have a critical gap, as many citizens are not able to take full advantage of their legal rights due to limited access and awareness of apposite legal information. This paper thus introduces Legal Assist AI, a highly efficient framework designed to provide legal assistance in the Indian domain. The core contribution is a framework demonstrating how a smaller, 8-billion-parameter quantized model (Llama 3.1) can achieve superior domain-specific performance. This effective performance stems from integrating a Retrieval-Augmented Generation (RAG) system with strategic prompt engineering, supported by a high-quality, up to date corpus of more than 600 legal documents. This corpus includes the Indian Constitution and more importantly, the newly enacted Bharatiya Nyaya Sanhita (BNS) and Bharatiya Nagarik Suraksha Sanhita (BNSS) among others. Further, by achieving a score of 60.08\% in the All-India Bar Examination (AIBE) benchmark, the specialized approach based on RAG was found to be highly efficient and effective, improving on the 58.72\% score of the 175-billion parameter GPT-3.5 Turbo. It was also observed that the framework was able to manage and mitigate instances of hallucinations successfully, which is a critical requirement for practical legal applications. A Parameter Efficiency Index (PEI) is also introduced, with the goal of quantifying the superior efficiency that the framework was able to achieve, demonstrating how the 8B model is 22 times more parameter-efficient than the 175B baseline, and hence corroborating the potential of smaller domain-adapted models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。