小模型答错法律题，靠加法条微调能显著提升准确率。
Do Small Models Use the Law You Give Them? Context-Injected Fine-Tuning for Legal QA in Bangladesh
- 用含法律条文的问答数据微调小模型，提升其使用检索法条的能力。
- 0.8B模型在英文考试中得分从2分升至34分，提升显著。
- 有效减少答案从孟加拉语跑偏到英语的现象，适合本地化法律AI研究者。
小语言模型即使获得法律条文，仍可能答错。我们测试在包含相关法律的示例上微调是否能提升后续对检索到法条的使用效果。我们从六部孟加拉国法案和三个附录中整理了2,165条双语问答数据，对Qwen3.5的0.8B、2B和4B参数量模型进行微调。评估基于2022年和2023年孟加拉国律师协会考试的孟加拉语及机器翻译英文版本，不使用检索，采用严格一致性评分，三次种子运行结果一致。0.8B模型在英文FAISS测试中得分从2提升至34/100。0.8B和2B模型的提升在配对测试中依然显著，但4B模型无明显净收益：孟加拉语表现提升，而多个英文条件反而下降。微调还使答案从孟加拉语漂移至纯英文的比例从44.0–53.2%降至0.2–0.7%，调整后p<0.001。因此，检索质量并非唯一瓶颈。小规模双语法律模型在使用提供法条方式及回答语言选择上存在差异。数据集已公开于https://huggingface.co/datasets/momahadi/bangladesh-legal-qa-dataset。
原文摘要 · Abstract (English)
A small language model can receive the governing statutory provision and still answer incorrectly. We test whether fine-tuning on examples containing relevant law improves later use of retrieved law. We curate 2{,}165 bilingual QA records from six Bangladeshi acts and three schedules, then fine-tune Qwen3.5 at 0.8B, 2B, and 4B. Evaluation uses the 2022 and 2023 Bangladesh Bar Council exams in Bangla and machine-translated English, with no retrieval, BM25, or FAISS, scored by strict consistency over three seeded runs. At 0.8B, fine-tuning raises the 2022 English FAISS score from 2 to 34 of 100. Gains at 0.8B and 2B survive paired testing, but the 4B model has no detectable net gain: Bangla improves while several English conditions regress. Fine-tuning also reduces answers that drift from Bangla into mostly English from 44.0--53.2\% to 0.2--0.7\%, with adjusted $p<.001$ at every scale. Retrieval quality is therefore not the only bottleneck. Small bilingual legal models also differ in how they use supplied law and whether they answer in the requested language. The dataset is publicly available at https://huggingface.co/datasets/momahadi/bangladesh-legal-qa-dataset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。