arXiv:2607.11193cs.SEcs.AI2026-07中稿 · the Research Track…

针对Transformer模型的修复难题,提出基于搜索的自动化修复方法。

RepTran: Search-Based Repair of Transformer Models

  • 结合方差与双向评分定位可疑权重,用差分进化迭代优化
  • 在18个故障基准上平均修复率达74.7%,优于现有方法
  • 专为Transformer结构设计,适合提升AI软件可靠性

为保障人工智能软件的整体质量,不仅传统软件组件需测试修复,人工智能组件同样需要。在各类AI组件中,Transformer模型日益融入软件系统,其异常行为具有重大影响。尽管已有软件工程研究提出深度神经网络(DNN)修复方法,但多数未考虑Transformer特有的结构。本文提出RepTran,一种面向Transformer模型的搜索式修复方法,聚焦其前馈网络(FFN),该模块在架构中起核心作用。RepTran通过结合基于方差的神经元得分与现有双向得分,识别可疑权重,并采用差分进化算法进行迭代优化。评估包含18个从CIFAR-100和Tiny-ImageNet构建的故障基准。对比三种基线:随机权重选择、当前最先进的DNN修复方法Arachne,以及可控制选择权重数的ArachneW。RepTran在所有基准上平均修复率达74.7%,统计上显著优于随机选择和Arachne。效应量分析显示,无论选取权重数量如何,RepTran修复率均高于ArachneW。结果表明RepTran能有效提升AI赋能软件的可靠性。

原文摘要 · Abstract (English)

To ensure the overall quality of AI-enabled software, not only traditional software components but also AI components need to be tested and repaired. Among AI components, Transformer models are increasingly integrated into software systems, which makes their misbehaviors critical. Although prior work in the software engineering community has proposed deep neural network (DNN) repair methods, most overlook Transformer-specific structures. We propose RepTran, a search-based repair method for Transformer models. It targets their feed-forward networks (FFNs), which play a central role in the architecture. RepTran identifies suspicious weights by combining two types of scores: a variance-based neuron score and an existing bidirectional score. It then iteratively optimizes these weights using differential evolution. Our evaluation includes 18 fault benchmarks constructed from CIFAR-100 and Tiny-ImageNet. We compare RepTran against three baselines: random weight selection, Arachne (a state-of-the-art DNN repair method), and ArachneW, which enables Arachne to control the number of selected weights. RepTran achieved an average repair rate of 74.7%, statistically outperforming random selection and Arachne across all benchmarks. Effect size analysis revealed that RepTran achieved higher repair rates than ArachneW regardless of the number of selected weights. These results suggest that RepTran is effective for enhancing the reliability of AI-enabled software.

Transformer模型修复搜索优化AI可靠性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。