arXiv:2602.00380cs.CL2026-02被引 2

对比大模型对土耳其语反身代词的绑定理解,发现本地绑定偏好差异显著。

Clause-Internal or Clause-External? Testing Turkish Reflexive Binding in Adapted versus Chain of Thought Large Language Models

  • 构建100句平衡语料,测试本地与远距离先行词的绑定选择
  • Trendyol-LLM在70%情况下选本地先行词,显示强局部偏好
  • OpenAI模型选择均衡,反映对语法局部性敏感度较低

本研究评估先进大模型对土耳其语反身代词绑定关系的理解能力。我们构建了一个包含100个土耳其语句子的平衡测试集,系统性地比较反身代词kendi和kendisi的本地与非本地先行词。对比两种模型:一个优化多步推理的OpenAI链式思维模型(o1 Mini)和一个基于LLaMA 2、在土耳其语数据上深度微调的Trendyol-LLM-7B-base-v0.1。通过结合句级困惑度与最小差异延续句的强制选择判断先行词选择。结果显示,Trendyol-LLM在约70%的试验中偏好本地绑定,表现出与结构邻近性一致的强局部性偏见;而OpenAI模型在本地与远距离读取间近乎平均分配选择,表明其对这种绑定配置的局部性敏感度较弱且不一致。结果揭示两模型在绑定行为上的显著差异,提示需深入分析模型架构、训练数据及推理策略如何影响土耳其语回指依赖的表征。

原文摘要 · Abstract (English)

This study evaluates whether state-of-the-art large language models capture the binding relations of Turkish reflexive pronouns. We construct a balanced evaluation set of 100 Turkish sentences that systematically pit local against non-local antecedents for the reflexives kendi and kendisi. We compare two contrasting systems: an OpenAI chain-of-thought model optimized for multi-step reasoning and Trendyol-LLM-7B-base-v0.1, a LLaMA 2 derived model extensively fine-tuned on Turkish data. Antecedent choice is assessed using a combined paradigm that integrates sentence-level perplexity with a forced-choice comparison between minimally differing continuations. Overall, Trendyol-LLM favors local bindings in approximately 70 percent of trials, exhibiting a robust locality bias consistent with a preference for structurally proximate antecedents. By contrast, the OpenAI model (o1 Mini) distributes its choices nearly evenly between local and long-distance readings, suggesting weaker or less consistent sensitivity to locality in this binding configuration. Taken together, these results reveal a marked contrast in binding behavior across the two systems and motivate closer analysis of how model architecture, training data, and inference-time reasoning strategies shape the representation of Turkish anaphoric dependencies.

语言模型语法理解反身代词土耳其语

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。