用相似查询搜索实现低资源语言意图分类的零样本方法
Intent Classification on Low-Resource Languages with Query Similarity Search
- 将意图分类转为查询相似性搜索,利用历史查询定义意图
- 在零样本设置下对低资源语言查询实现合理分类性能
- 适合需要快速扩展多语言支持的检索系统开发者
意图分类是信息检索系统的关键组成部分。现有方法通常将其视为分类问题,但意图难以定义,导致标注数据难获取且成本高。这一问题在多语言尤其是低资源语言场景下尤为突出。为此,我们提出将意图分类重构为查询相似性搜索任务:通过历史查询定义意图,并基于潜在空间中的查询相似度,根据最相似查询的标签来分类新查询。该方法在零样本条件下,实现了对低资源语言查询的合理意图分类性能。
原文摘要 · Abstract (English)
Intent classification is an important component of a functional Information Retrieval ecosystem. Many current approaches to intent classification, typically framed as a classification problem, can be problematic as intents are often hard to define and thus data can be difficult and expensive to annotate. The problem is exacerbated when we need to extend the intent classification system to support multiple and in particular low-resource languages. To address this, we propose casting intent classification as a query similarity search problem - we use previous example queries to define an intent, and a query similarity method to classify an incoming query based on the labels of its most similar queries in latent space. With the proposed approach, we are able to achieve reasonable intent classification performance for queries in low-resource languages in a zero-shot setting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。