用引导式提问补全信息,提升文本分类准确率
GUIDEQ: Framework for Guided Questioning for progressive informational collection and classification
- 基于分类器关键词与LLM生成精准引导问题
- 在医疗等场景下实现更高F1分数的分类性能
- 适合需高精度信息收集的应用场景
问答系统在医疗、客服、法律等领域用于信息收集与分类,但大模型常因知识依赖导致信息不足。本文提出GUIDEQ框架,通过分析分类器的显著性关键词(基于特征遮蔽),结合前3类预测结果,设计引导式提问策略,主动获取关键信息以弥补初始输入的不足。该方法能更精准地收集相关数据,从而提升文本分类效果。实验表明,相较于其他基于LLM的基线方法,GUIDEQ在多个数据集上获得更高的F1分数,并通过多轮分析验证了问题质量与可解释性优势。
原文摘要 · Abstract (English)
Question Answering (QA) is an important part of tasks like text classification through information gathering. These are finding increasing use in sectors like healthcare, customer support, legal services, etc., to collect and classify responses into actionable categories. LLMs, although can support QA systems, they face a significant challenge of insufficient or missing information for classification. Although LLMs excel in reasoning, the models rely on their parametric knowledge to answer. However, questioning the user requires domain-specific information aiding to collect accurate information. Our work, GUIDEQ, presents a novel framework for asking guided questions to further progress a partial information. We leverage the explainability derived from the classifier model for along with LLMs for asking guided questions to further enhance the information. This further information helps in more accurate classification of a text. GUIDEQ derives the most significant key-words representative of a label using occlusions. We develop GUIDEQ's prompting strategy for guided questions based on the top-3 classifier label outputs and the significant words, to seek specific and relevant information, and classify in a targeted manner. Through our experimental results, we demonstrate that GUIDEQ outperforms other LLM-based baselines, yielding improved F1-Score through the accurate collection of relevant further information. We perform various analytical studies and also report better question quality compared to our method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。