arXiv:2503.11302cs.CL2025-03被引 12

发现大模型处理语法与语义任务时机制分离,但语法内部也无统一网络。

Are formal and functional linguistic mechanisms dissociated in language models?

  • 通过分析最小计算子图,对比不同任务的内在机制
  • 语法与语义任务间电路重叠极低,语法任务间也无统一网络
  • 适合关注模型内部表征结构的研究者参考

尽管大语言模型(LLMs)能力持续提升,但其表现不均衡:在生成流畅、语法正确的文本等形式语言任务上表现优异,但在推理和一致的事实检索等功能性语言任务上仍较弱。受神经科学启发,近期研究认为,要同时胜任两类任务,模型应为每类任务使用不同的机制;这种功能分化可能内置或在训练中自发形成。本文通过识别并比较负责各类形式与功能性任务的“电路”(即最小计算子图),探究当前模型是否已出现这种分化。在5个主流模型、10个不同任务上的比较表明,形式与功能性任务间的电路几乎无重叠,且形式任务内部也缺乏统一网络,因此尚未发现一个独立于功能任务的单一形式语言网络。然而,在跨任务可迁移性(即一个电路能否完成另一任务)方面,形式与功能机制之间存在明显分离,暗示形式任务间可能存在共享机制。

原文摘要 · Abstract (English)

Although large language models (LLMs) are increasingly capable, these capabilities are unevenly distributed: they excel at formal linguistic tasks, such as producing fluent, grammatical text, but struggle more with functional linguistic tasks like reasoning and consistent fact retrieval. Inspired by neuroscience, recent work suggests that to succeed on both formal and functional linguistic tasks, LLMs should use different mechanisms for each; such localization could either be built-in or emerge spontaneously through training. In this paper, we ask: do current models, with fast-improving functional linguistic abilities, exhibit distinct localization of formal and functional linguistic mechanisms? We answer this by finding and comparing the "circuits", or minimal computational subgraphs, responsible for various formal and functional tasks. Comparing 5 LLMs across 10 distinct tasks, we find that while there is indeed little overlap between circuits for formal and functional tasks, there is also little overlap between formal linguistic tasks, as exists in the human brain. Thus, a single formal linguistic network, unified and distinct from functional task circuits, remains elusive. However, in terms of cross-task faithfulness - the ability of one circuit to solve another's task - we observe a separation between formal and functional mechanisms, suggesting that shared mechanisms between formal tasks may exist.

语言模型机制分析神经科学启发

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。