用大模型自动推导语言演变规律，提升古语言研究效率
Programming by Examples Meets Historical Linguistics: A Large Language Model Based Approach to Sound Law Induction
- 将音变规律归纳转化为示例编程问题，利用大模型求解
- 新方法在测试中通过率提升6%，参数量仅为次优模型的1/3
- 适合语言学研究者与自动化推理方向的开发者
历史语言学家长期依赖手动编写规则（即音变规律），将祖先语言词汇转换为现存后代词形。这一过程耗时费力，促使人们探索自动化音变规律推导（SLI）。本文首次将该问题建模为基于大语言模型（LLM）的示例编程（PBE）任务。尽管大模型在代码生成方面表现优异，但此前研究表明，示例编程仍具挑战性，且可通过微调改善，尤其当训练数据与评估数据分布相似时。本文构建了针对SLI任务的“分布相似性”概念框架，提出四种带有不同归纳偏置的合成数据生成方法，以探究最优性能来源。基于实验结果，我们构建了一个当前最优的开源音变规律推导模型（作为PBE任务求解器），相较第二佳方案提升6%通过率，同时仅需其三分之一参数量。研究还揭示了示例编程未来的重要发展方向。
原文摘要 · Abstract (English)
Historical linguists have long written "programs" that convert reconstructed words in an ancestor language into their attested descendants via ordered string rewrite functions (called sound laws) However, writing these programs is time-consuming, motivating the development of automated Sound Law Induction (SLI) which we formulate as Programming by Examples (PBE) with Large Language Models (LLMs) in this paper. While LLMs have been effective for code generation, recent work has shown that PBE is challenging but improvable by fine-tuning, especially with training data drawn from the same distribution as evaluation data. In this paper, we create a conceptual framework of what constitutes a "similar distribution" for SLI and propose four kinds of synthetic data generation methods with varying amounts of inductive bias to investigate what leads to the best performance. Based on the results we create a SOTA open-source model for SLI as PBE (+6% pass rate with a third of the parameters of the second-best LLM) and also highlight exciting future directions for PBE research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。