改进句子编码器微调方法，提升虚拟助手的意图识别与范围外查询检测能力
A new approach for fine-tuning sentence transformers for intent classification and out-of-scope detection tasks
- 在交叉熵损失基础上加入自编码器重建损失，约束正常查询嵌入分布
- 在未知范围外数据下，拒识效果提升1-4%（AUPRC）
- 适合需要高精度范围外检测的智能客服、虚拟助手系统
在虚拟助手系统中，准确拒绝或转接超出服务范围的用户查询至关重要。当前最有效的方法是将意图分类与范围外（OOS）拒绝联合建模，并利用基于Transformer的句子编码器生成的嵌入相似性进行判断。通常这些编码器通过交叉熵损失微调用于意图分类任务，但近期研究表明，该方法会使正常查询的嵌入在嵌入空间中过度分散，导致与范围外嵌入发生重叠，从而增加拒识难度，尤其当范围外数据未知时更为严重。为解决此问题，本文提出在交叉熵损失中引入基于自编码器学习的正常查询嵌入重建损失作为正则项。实验表明，该方法在不损害意图分类性能的前提下，使范围外实例的拒识效果在精确率-召回率曲线下面积（AUPRC）上提升1%-4%。
原文摘要 · Abstract (English)
In virtual assistant (VA) systems it is important to reject or redirect user queries that fall outside the scope of the system. One of the most accurate approaches for out-of-scope (OOS) rejection is to combine it with the task of intent classification on in-scope queries, and to use methods based on the similarity of embeddings produced by transformer-based sentence encoders. Typically, such encoders are fine-tuned for the intent-classification task, using cross-entropy loss. Recent work has shown that while this produces suitable embeddings for the intent-classification task, it also tends to disperse in-scope embeddings over the full sentence embedding space. This causes the in-scope embeddings to potentially overlap with OOS embeddings, thereby making OOS rejection difficult. This is compounded when OOS data is unknown. To mitigate this issue our work proposes to regularize the cross-entropy loss with an in-scope embedding reconstruction loss learned using an auto-encoder. Our method achieves a 1-4% improvement in the area under the precision-recall curve for rejecting out-of-sample (OOS) instances, without compromising intent classification performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。