构建多语言数据集,提升对话式AI解释系统对自定义输入的跨语言理解能力。
Multilingual Datasets for Custom Input Extraction and Explanation Requests Parsing in Conversational XAI Systems
- 扩展CoXQL为五种语言的多语言数据集,含一种低资源语言。
- 提出新解析方法,在多语言环境下显著提升意图识别准确率。
- 设计新数据集Compass,支持用户自定义输入的跨语言解析,适合多语言AI研究者。
基于大语言模型的对话式可解释人工智能(ConvXAI)系统因能通过对话增强用户理解而备受关注。现有系统通常依赖意图识别来精准判断用户需求并匹配解释方法,虽在英语中表现优异,但在多语言场景下受限于训练数据稀缺,泛化能力不足。此外,对用户自定义输入(即非预设数据集实例的自由格式输入)的支持仍十分有限。为此,我们首先引入MultiCoXQL,一个涵盖五种类型差异显著语言(包括一种低资源语言)的CoXQL数据集扩展版本。随后,提出一种新解析方法以提升多语言解析性能,并在MultiCoXQL上评估三种LLM及多种解析策略。此外,我们构建了Compass——一个专用于ConvXAI系统中自定义输入提取的多语言数据集,覆盖11种意图,同样包含五种语言。我们在Compass上开展单语、跨语言和多语言评估,使用三种不同规模的LLM及BERT类模型进行测试。
原文摘要 · Abstract (English)
Conversational explainable artificial intelligence (ConvXAI) systems based on large language models (LLMs) have garnered considerable attention for their ability to enhance user comprehension through dialogue-based explanations. Current ConvXAI systems often are based on intent recognition to accurately identify the user's desired intention and map it to an explainability method. While such methods offer great precision and reliability in discerning users' underlying intentions for English, a significant challenge in the scarcity of training data persists, which impedes multilingual generalization. Besides, the support for free-form custom inputs, which are user-defined data distinct from pre-configured dataset instances, remains largely limited. To bridge these gaps, we first introduce MultiCoXQL, a multilingual extension of the CoXQL dataset spanning five typologically diverse languages, including one low-resource language. Subsequently, we propose a new parsing approach aimed at enhancing multilingual parsing performance, and evaluate three LLMs on MultiCoXQL using various parsing strategies. Furthermore, we present Compass, a new multilingual dataset designed for custom input extraction in ConvXAI systems, encompassing 11 intents across the same five languages as MultiCoXQL. We conduct monolingual, cross-lingual, and multilingual evaluations on Compass, employing three LLMs of varying sizes alongside BERT-type models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。