用英文释义作锚点，评估跨语言习语对齐效果
G-IdiomAlign: A Gloss-Pivoted Benchmark for Cross-Lingual Idiom Alignment
- 以维基词典释义为语义锚点构建评测基准
- 释义能显著提升生成质量，但整体表现仍有提升空间
- 适合研究多语言习语理解与生成的AI工程师
习语因非构式性和弱表层对应，跨语言迁移困难，直译不可靠。本文提出G-IdiomAlign，一个以英文释义为语义锚点的基准，每个习语均来自维基词典。我们构建了高置信度参考对齐集，支持可复现评估。该基准包含两种评测协议：(1) 带类型干扰项的多选习语等价测试，用于错误归因；(2) 释义对比生成任务，比较无释义与有释义输入下模型表现，以分离显式语义锚的作用。在多种大模型上，模型普遍倾向字面翻译，尤其在低资源语言中更明显。有释义时，基于嵌入的语义代理指标显示生成性能一致提升，但绝对表现仍有限，表明开放输出空间中仍有巨大改进空间。对Qwen3-8B的进一步分析发现，条件间差异主要集中在注意力头而非层，且高质量有释义生成与更强的释义锚定相关。
原文摘要 · Abstract (English)
Idioms are difficult to transfer across languages due to their non-compositionality and weak surface-form grounding, making literal mappings unreliable. We present G-IdiomAlign, a gloss-pivoted benchmark where each idiom is anchored by an English gloss from Wiktionary. We further construct a high-confidence reference alignment set for reproducible evaluation. G-IdiomAlign supports two protocols: (1) a controlled Multiple-Choice Idiom Equivalence with typed distractors for error attribution; and (2) a Gloss-Contrastive Generation contrasting No-gloss and With-gloss inputs to isolate the effect of an explicit semantic pivot. Across diverse LLMs, a bias to literal translation is a dominant failure mode, especially when the target is a low-resource language. Glosses consistently improve Gloss-Contrastive Generation under an embedding-based semantic proxy, but performance remains modest, indicating substantial headroom in the open output space. Subsequent analysis on Qwen3-8B further suggests that cross-condition differences are concentrated more in attention heads than in layers, while better With-gloss generations coincide with stronger gloss anchoring.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。