用微调的多语言Transformer提升IT工单优先级判断准确率
Learning to Prioritize IT Tickets: A Comparative Evaluation of Embedding-based Approaches and Fine-Tuned Transformer Models
- 用Transformer同时处理文本和数值特征，比传统嵌入方法更优
- 平均F1达78.5%，加权Cohen's kappa接近0.80，表现显著提升
- 适合需要高精度工单分类的运维团队或自动化系统开发者
在IT服务管理中，工单优先级排序对运营效率至关重要，但受文本噪声、写作风格主观性及类别极度不平衡等因素影响，仍具挑战。本文对比两类方法：基于嵌入的流水线（含降维、聚类与经典分类器）和微调的多语言Transformer模型（可处理文本与数值特征）。嵌入方法在30种配置下泛化能力差，聚类无法发现有效结构，监督模型对嵌入质量高度敏感。相比之下，所提Transformer模型表现显著更优，平均F1得分为78.5%，加权Cohen's kappa值接近0.80，表明与真实标签高度一致。结果揭示通用嵌入在ITSM数据上的局限性，并验证领域适配的Transformer架构在工单优先级排序中的有效性。
原文摘要 · Abstract (English)
Prioritizing service tickets in IT Service Management (ITSM) is critical for operational efficiency but remains challenging due to noisy textual inputs, subjective writing styles, and pronounced class imbalance. We evaluate two families of approaches for ticket prioritization: embedding-based pipelines that combine dimensionality reduction, clustering, and classical classifiers, and a fine-tuned multilingual transformer that processes both textual and numerical features. Embedding-based methods exhibit limited generalization across a wide range of thirty configurations, with clustering failing to uncover meaningful structures and supervised models highly sensitive to embedding quality. In contrast, the proposed transformer model achieves substantially higher performance, with an average F1-score of 78.5% and weighted Cohen's kappa values of nearly 0.80, indicating strong alignment with true labels. These results highlight the limitations of generic embeddings for ITSM data and demonstrate the effectiveness of domain-adapted transformer architectures for operational ticket prioritization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。