arXiv:2603.09978cs.SEcs.AI2026-03

用少量参数提升代码分析多任务性能,效果媲美全量微调。

One Model, Many Skills: Parameter-Efficient Fine-Tuning for Multitask Code Analysis

  • 共享一个小型参数模块实现多任务微调,大幅降低计算开销。
  • 相比全量微调,参数量减少任务数倍,计算成本降低85%。
  • 适合资源有限但需兼顾多种代码分析任务的开发者使用。

大语言模型在代码生成上已超越专用系统,但在其他代码分析任务上的表现仍不明确。多任务学习可统一多个目标于单一模型,但全量微调大模型成本过高。参数高效微调(PEFT)通过仅更新少量权重缓解此问题。尽管PEFT在单任务中表现良好,其在多任务中的潜力尚未系统探索。本文首次全面评估代码分析领域的多任务PEFT,对比多种方法在不同任务与模型架构下的表现。实验表明,单一共享的PEFT模块在多数情况下可达到甚至超过全量多任务微调的性能,验证了PEFT在多任务中的有效性。相较于单任务设置,多任务PEFT实现了优异的性能-效率权衡:精度接近单任务微调,同时将可训练参数数量减少为任务数的倒数,计算成本最高降低85%。此外,多任务收益对任务分组敏感,任务稳定性、模型架构、互补性、不对称性及数据质量是决定共微调成败的关键因素。最后,我们将高效多任务PEFT与DeepSeek、Qwen、Mistral、CodeLlama和StarCoder等开源通用大模型的直接提示进行对比。尽管这些模型在代码生成中表现优异,但在分析任务上表现较差;即使10亿参数的模型经多任务PEFT后,结果也显著更优。

原文摘要 · Abstract (English)

Large language models have recently surpassed specialized systems on code generation, yet their effectiveness on other code-analysis tasks remains less clear. At the same time, multi-task learning offers a way to unify diverse objectives within a single model, but fully fine-tuning LLMs across tasks is computationally prohibitive. Parameter-efficient fine-tuning mitigates this cost by updating only a small fraction of weights. Although PEFT has proven effective in single-task settings, its potential for multi-task learning has not yet been systematically explored. We present the first comprehensive evaluation of multi-task PEFT for code analysis, comparing several methods across diverse tasks and model architectures. Our experiments show that a single PEFT module shared across tasks can match, and in some cases surpass, full multi-task fine-tuning, confirming that the benefits of PEFT extend beyond isolated tasks. When comparing single-task and multi-task setups, we find that multi-task PEFT achieves a favorable performance-efficiency trade-off: it delivers accuracy close to single-task fine-tuning while reducing storage requirements, cutting the number of trainable parameters by a factor of the task count, and lowering computation costs by as much as 85%. At the same time, multi-task gains remain sensitive to task grouping. Through task-pairing experiments, we identify key factors shaping outcomes: task stability, model architecture, task complementarity, asymmetry, and dataset quality determine the success of co-fine-tuning. Finally, we benchmark efficient multi-task PEFT against direct prompting of open-source general-purpose LLMs, including DeepSeek, Qwen, Mistral, CodeLlama, and StarCoder. Despite their strong performance in code generation, these models underperform on analysis tasks, where even a 1B-parameter model with multi-task PEFT achieves significantly better results.

代码分析参数高效多任务学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。