LLM在正向与逆向科学文本上表现相当，说明其成功不依赖人类语言机制。
Beyond Human-Like Processing: Large Language Models Perform Equivalently on Forward and Backward Scientific Text
- 用正向和逆向科学文本训练LLM，测试其处理能力
- 在神经科学基准上，模型表现超越人类专家，正逆向无差异
- 适合研究模型本质、警惕将语言任务成功归因于类人机制的读者
大型语言模型（LLMs）的卓越表现使其被视为人类语言处理的模型。然而，我们提出，其成功源于Transformer学习架构的灵活性。为验证这一假说，我们在正向或逆向格式的科学文本上训练了LLMs。尽管逆向文本不符合人类语言结构，但模型在正向与逆向文本上的表现相当，并在神经科学基准上超越了人类专家表现。结果表明，Transformer在多个领域（如天气预测、蛋白质设计）的成功，源于其从任何足够结构化的输入中提取预测模式的能力。鉴于其通用性，应谨慎将LLM在语言任务中的成功解释为人类类比机制的证据。
原文摘要 · Abstract (English)
The impressive performance of large language models (LLMs) has led to their consideration as models of human language processing. Instead, we suggest that the success of LLMs arises from the flexibility of the transformer learning architecture. To evaluate this conjecture, we trained LLMs on scientific texts that were either in a forward or backward format. Despite backward text being inconsistent with the structure of human languages, we found that LLMs performed equally well in either format on a neuroscience benchmark, eclipsing human expert performance for both forward and backward orders. Our results are consistent with the success of transformers across diverse domains, such as weather prediction and protein design. This widespread success is attributable to LLM's ability to extract predictive patterns from any sufficiently structured input. Given their generality, we suggest caution in interpreting LLM's success in linguistic tasks as evidence for human-like mechanisms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。