arXiv:2601.22264cs.SEcs.AI2026-01中稿 · the ACM Internatio…被引 1

用少量日志快速分类间歇性失败,提升诊断效率。

Predicting Intermittent Job Failure Categories for Diagnosis Using Few-Shot Fine-Tuned Language Models

  • 基于预训练语言模型的少样本学习,仅需12条标注日志
  • 宏平均F1达84.3%,前2名准确率92.0%
  • 秒级定位关键日志,减少74.4%人工审查工作量

持续集成(CI)流水线失败本应为开发者提供代码错误反馈,但因测试非确定性、网络中断、基础设施故障、资源耗尽等可靠性问题,常出现间歇性失败。这类失败导致大量计算资源浪费和诊断时间消耗,干扰开发核心任务,常需专业团队介入。现有研究虽能检测间歇性失败,但未解决后续诊断难题。为此,本文提出FlaXifyer,一种基于预训练语言模型的少样本分类方法,仅需作业执行日志即可实现84.3%的宏平均F1和92.0%的Top-2准确率,每类仅需12个标注样本。同时提出LogSift可解释性技术,能在1秒内识别关键日志,使审查工作量减少74.4%,并在87%的案例中有效揭示故障信息。在TELUS的2,458个失败案例上验证表明,FlaXifyer与LogSift能实现高效自动归类,加速故障诊断,推动间歇性失败的自动化解决。

原文摘要 · Abstract (English)

In principle, Continuous Integration (CI) pipeline failures provide valuable feedback to developers on code-related errors. In practice, however, pipeline jobs often fail intermittently due to non-deterministic tests, network outages, infrastructure failures, resource exhaustion, and other reliability issues. These intermittent (flaky) job failures lead to substantial inefficiencies: wasted computational resources from repeated reruns and significant diagnosis time that distracts developers from core activities and often requires intervention from specialized teams. Prior work has proposed machine learning techniques to detect intermittent failures, but does not address the subsequent diagnosis challenge. To fill this gap, we introduce FlaXifyer, a few-shot learning approach for predicting intermittent job failure categories using pre-trained language models. FlaXifyer requires only job execution logs and achieves 84.3% Macro F1 and 92.0% Top-2 accuracy with just 12 labeled examples per category. We also propose LogSift, an interpretability technique that identifies influential log statements in under one second, reducing review effort by 74.4% while surfacing relevant failure information in 87% of cases. Evaluation on 2,458 job failures from TELUS demonstrates that FlaXifyer and LogSift enable effective automated triage, accelerate failure diagnosis, and pave the way towards the automated resolution of intermittent job failures.

故障诊断少样本学习日志分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。