用大模型+人工协作,自动发现服务反馈中的新问题和不公平现象。
LLM-based Models for Detecting Emerging Topics in Service Feedback

- 结合微调量化大模型与专家审核,实现多语言反馈分析。
- 相比基线模型,生成结果与税务专家判断更一致。
- 适合公共部门做可信赖的、大规模服务优化决策。
提升服务反馈分析对公共部门至关重要,尤其在税务机构中,信任与合规依赖于公平有效的服务交付。随着反馈量增长,识别新兴服务质量问题及不同人群间的潜在差异日益困难。传统方法多依赖人工审查或静态专家定义指标,难以扩展且难以捕捉文本中的复杂模式。本文提出一种融合大语言模型(LLMs)、统计技术与人机协作的新方法,旨在检测可能揭示服务交付不公的新兴质量话题。框架采用微调并量化的LLM结合专家监督,实现准确、高效且上下文敏感的分析。通过相似性分析及资深税务官评估,验证了该方法在专家判断一致性上优于基线模型。引入人机协同机制,降低大模型幻觉,提升洞察的可靠性与相关性。结果表明,结合大模型与人类经验,可支持公共部门实现可扩展、基于证据的决策。本研究推动负责任AI系统发展,通过更有效的多语言客户反馈分析,提升服务品质、响应速度、公平性与公众信任。
原文摘要 · Abstract (English)
Enhancing the analysis of service feedback is essential for public sector organizations, particularly tax administrations, where trust and compliance depend on fair and effective service delivery. As feedback volumes grow, identifying emerging service quality issues and potential disparities across diverse populations becomes increasingly challenging. Traditional approaches often rely on manual review or static expert-defined indicators, limiting scalability and the ability to capture complex patterns in textual feedback. This paper presents a novel methodology that integrates large language models (LLMs), statistical techniques, and human-AI collaboration to improve multilingual customer feedback analysis. The primary objective is to detect emerging service quality topics that may also reveal potential inequities in service delivery. Our framework combines fine-tuned, quantized LLMs with expert oversight to produce accurate, computationally efficient, and context-aware analyses. The proposed approach was evaluated using similarity analysis and assessments from experienced tax officers, demonstrating stronger alignment with expert judgments than baseline models. By incorporating a human-in-the-loop framework, the methodology reduces LLM fabrication while improving the reliability and relevance of generated insights. The results demonstrate the practicality of combining LLMs with human expertise to support scalable, evidence-based decision-making in public sector organizations. This work contributes to the development of responsible AI systems that enhance service quality, responsiveness, fairness, and public trust through more effective analysis of multilingual customer feedback.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。