测试多语言下大模型调用工具的鲁棒性，发现语言错配导致执行失败。
Lost in Execution: On the Multilingual Robustness of Tool Calling in Large Language Models
- 构建MLCL基准，评估中、印地、伊戈博语下的工具调用能力
- 即使理解意图并选对工具，仍因参数语言不一致导致失败
- 现有方法能降错但难恢复英语水平性能，适合多语言应用研究者
大型语言模型（LLMs）越来越多地作为智能体，通过结构化函数调用外部工具。尽管近期研究在以英语为中心的评测中报告了出色的工具调用表现，但多语言用户交互下的鲁棒性仍缺乏深入探索。本文提出MLCL诊断基准，系统评估中文、印地语和低资源语言伊戈博语中的多语言工具调用表现。细粒度错误分析显示，许多失败发生在意图理解与工具选择正确的情况下。我们识别出参数值语言不匹配是主要失败模式：模型虽在用户语言中生成语义正确的参数值，却违反了语言无关的执行规范。进一步评估多种推理时系统策略，发现它们虽显著降低语言相关执行错误，但均无法完全恢复英语水平性能。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are increasingly deployed as agents that invoke external tools through structured function calls. While recent work reports strong tool-calling performance under standard English-centric evaluations, the robustness of tool calling under multilingual user interactions remains underexplored. In this work, we introduce MLCL, a diagnostic benchmark, and conduct a systematic evaluation of multilingual tool calling across Chinese, Hindi, and the low-resource language Igbo. Through fine-grained error analysis, we show that many failures occur despite correct intent understanding and tool selection. We identify parameter value language mismatch as a dominant failure mode, where models generate semantically appropriate parameter values in the user's language, violating language-invariant execution conventions. We further evaluate several inference-time system strategies and find that while these strategies substantially reduce language-induced execution errors, none of them can fully recover English-level performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。