大模型能解语言奥赛题，还能自动生成新题，推动语言学普及。
Can LLMs Generate and Solve Linguistic Olympiad Puzzles?
- 用大模型解语言奥赛题，跨语言主题表现优异
- 在多数题型上超越人类，唯书写系统类例外
- 首次实现自动出题，助力小众语言传播
本文提出并探索了语言奥赛谜题的求解与生成任务。我们扩展了现有求解基准，评估了包括OpenAI o1在内的大语言模型在多种语言主题上的表现。实验表明，大模型在大多数题型上优于人类，但在涉及书写系统的题目以及未充分研究的语言中表现较弱。基于解题结果，我们设计了新颖的谜题生成任务。我们认为，自动化生成语言谜题——即使仅限简单题目——也有助于激发公众对语言学的兴趣，推动冷门语言知识的传播。该研究凸显了语言谜题生成作为重要科研方向的价值。
原文摘要 · Abstract (English)
In this paper, we introduce a combination of novel and exciting tasks: the solution and generation of linguistic puzzles. We focus on puzzles used in Linguistic Olympiads for high school students. We first extend the existing benchmark for the task of solving linguistic puzzles. We explore the use of Large Language Models (LLMs), including recent state-of-the-art models such as OpenAI's o1, for solving linguistic puzzles, analyzing their performance across various linguistic topics. We demonstrate that LLMs outperform humans on most puzzles types, except for those centered on writing systems, and for the understudied languages. We use the insights from puzzle-solving experiments to direct the novel task of puzzle generation. We believe that automating puzzle generation, even for relatively simple puzzles, holds promise for expanding interest in linguistics and introducing the field to a broader audience. This finding highlights the importance of linguistic puzzle generation as a research task: such puzzles can not only promote linguistics but also support the dissemination of knowledge about rare and understudied languages.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。