构建美国移民法问答数据集并微调小模型，辅助非律师群体理解复杂法规。
ImmigrationQA: A Source-Grounded Dataset and Small-Model Adaptation for U.S. Immigration Law
- 基于13个子领域1.7万组问题，从官方文件生成精准问答对。
- 微调后的30亿参数模型在关键任务上准确率提升27%，达16.8%完全正确。
- 适合法律援助、社区咨询等低资源场景，但不能替代专业律师意见。
美国移民法涵盖数千页官方政策、联邦法规和程序指南，频繁更新且对无律师代理的申请人影响重大。本文构建了ImmigrationQA，一个包含17,058对问答的源文本支撑型数据集，覆盖13个移民子领域，并使用参数高效微调方法LoRA对Llama 3.2 3B Instruct模型进行训练。数据源自11个一、二级权威来源，包括USCIS政策手册、8 CFR、BIA判例及社区问答，共生成10,056份经验证的规范文档与18,308个文本片段。通过Claude Sonnet 4.6结合五种特定模式提示词生成结构化问答对，剔除22对源覆盖不足的样本。在993对保留测试集上，采用大模型作为评判者对101例分层样本评估，微调模型平均得分为1.08/3.0（16.8%完全正确），优于基线模型Llama 3 8B的0.85/3.0（4%完全正确），相对提升27%；零样本Claude Sonnet得分为1.52/3.0（25%完全正确）。模型在旅行文件、身份调整、非移民签证等程序性领域表现显著，但在复杂法律推理和时效性数据上仍较弱。整个流程云算力成本约29美元。所有成果——数据集、模型、代码及提示模板——均已公开。系统不替代法律咨询，亦不反映爬取日期后的法规变更。
原文摘要 · Abstract (English)
U.S. immigration law spans thousands of pages of official policy, federal regulations, and procedural guidance that change frequently and carry high stakes for petitioners who lack legal representation. We describe the construction of ImmigrationQA, a source-grounded question-answering dataset of 17,058 pairs across 13 immigration subdomains, and the fine-tuning of a Llama 3.2 3B Instruct model on that dataset using parameter-efficient LoRA. The corpus was assembled from 11 primary and secondary sources -- including the USCIS Policy Manual, 8 CFR, BIA precedent decisions, and community Q&A -- yielding 10,056 validated canonical documents and 18,308 text chunks. Structured QA pairs were generated from these chunks using Claude Sonnet 4.6 via five mode-specific prompts, with 22 pairs rejected for insufficient source-span overlap. The fine-tuned model was evaluated against a held-out split of 993 pairs using LLM-as-judge scoring on a 101-example stratified sample. The fine-tuned model scored a mean of 1.08/3.0 (16.8% fully correct; 101-example stratified eval) versus the Llama 3 8B base model at 0.85/3.0 (4% fully correct), a relative improvement of 27% in mean score; a zero-shot Claude Sonnet baseline scored 1.52/3.0 (25% fully correct). The fine-tuned model shows concentrated improvement in procedural subdomains (travel documents, adjustment of status, nonimmigrant visas) while remaining weak on complex legal reasoning and time-sensitive statistics. The full pipeline ran for approximately $29 in cloud compute. All artifacts -- dataset, model, code, and prompt templates -- are publicly released. The system is not a substitute for legal counsel and does not reflect regulatory changes after the corpus crawl date.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。