用非线性朴素贝叶斯+注意力机制，让聊天模型不跑题。
Don't Shoot The Breeze: Topic Continuity Model Using Nonlinear Naive Bayes With Attention
- 基于朴素贝叶斯扩展为可量化公式，引入注意力与对数非线性增强连续性判断。
- 处理任意长度对话，时间复杂度线性，比传统方法更稳定高效。
- 适合需要可解释性、长对话连贯性的LLM应用场景。
在多样商业场景中使用大语言模型（LLM）作为聊天机器人时，保持话题连贯性常面临挑战。话题突变会导致用户体验下降及计算资源浪费。本文提出一种话题连续性评估模型，用于判断回复是否与初始话题一致。该模型基于自然语言理解（NLU）模型，通过朴素贝叶斯方法扩展为可量化的表达式，并引入注意力机制与对数非线性以增强捕捉话题连续性的能力。该方法将NLU模型转化为可解释的分析公式。相比受限于词元数量的传统模型，本模型可无缝处理任意长度对话，具有线性时间复杂度。实验表明，该模型在长且复杂的对话中显著优于传统方法。这一特性为实现负责任且可解释的LLM应用提供了可能。
原文摘要 · Abstract (English)
Utilizing Large Language Models (LLM) as chatbots in diverse business scenarios often presents the challenge of maintaining topic continuity. Abrupt shifts in topics can lead to poor user experiences and inefficient utilization of computational resources. In this paper, we present a topic continuity model aimed at assessing whether a response aligns with the initial conversation topic. Our model is built upon the expansion of the corresponding natural language understanding (NLU) model into quantifiable terms using a Naive Bayes approach. Subsequently, we have introduced an attention mechanism and logarithmic nonlinearity to enhance its capability to capture topic continuity. This approach allows us to convert the NLU model into an interpretable analytical formula. In contrast to many NLU models constrained by token limits, our proposed model can seamlessly handle conversations of any length with linear time complexity. Furthermore, the attention mechanism significantly improves the model's ability to identify topic continuity in complex conversations. According to our experiments, our model consistently outperforms traditional methods, particularly in handling lengthy and intricate conversations. This unique capability offers us an opportunity to ensure the responsible and interpretable use of LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。