1.8B参数模型模拟日语转英语的二语习得，避免过早接触英语。
Dango: A Strictly L1-Only Large Language Model for Studying Second Language Acquisition
- 仅用日语语料预训练，通过过滤减少英语污染。
- 在生成的英语学习材料上微调后，产出类人英语表达。
- 适合研究语言迁移机制或开发教学工具。
我们提出 Dango，一个 1.8B 参数的大语言模型，用于受控研究日语到英语的二语习得（SLA）。以往研究多依赖小规模或非解码器模型，难以生成开放式文本，限制其作为真实二语模拟器的能力。我们发现，大规模模型面临关键挑战：用于一语习得的“单语”预训练语料中存在英语污染。为此，我们提出一种过滤方法，在保留适度英语暴露的同时减少过早接触。随后，我们在大语言模型生成的英语学习课程上对模型进行微调，以模拟二语习得过程。评估表明，Dango 发展出类人二语输出模式，优于未过滤及标准多语基线。我们公开模型、数据与代码，推动可复现的计算型二语习得研究及面向学习者的应用。
原文摘要 · Abstract (English)
We introduce Dango, a 1.8B-parameter large language model designed for controlled studies of L1-to-L2 (Japanese-to-English) transfer in second language acquisition (SLA). While previous studies have explored SLA in language models, they have predominantly relied on smaller or non-decoder models, limiting their ability to generate open-ended text and reducing their suitability as practical L2 simulators. We identify a key challenge when scaling models to this size: L2 contamination within the "monolingual" pretraining corpus used for L1 acquisition. To address this, we propose a filtering method to reduce premature exposure to English while preserving realistic, minimal exposure. We then fine-tune the model on LLM-generated L2-learning lessons to simulate the L2 acquisition process. Our evaluations confirm that Dango develops human-like L2 production patterns, outperforming both unfiltered and standard multilingual baselines. We release the model, data, and code to facilitate reproducible computational SLA research and learner-facing applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。