arXiv:2608.21208cs.SEcs.AI2026-08

研究大模型迁移代码时规格书的跨工具兼容性,发现不同工具理解相同规格差异巨大。

Specification Portability Across LLM Development Agents: Cross-Agent Compatibility in Specification-Driven Software Migration

  • 以Oracle到PostgreSQL迁移为任务,测试多个大模型对规格书的解读能力。
  • 部分模型直接使用其他工具生成的规格,语法正确率低至2.33%,相似度不足0.015。
  • 检索增强输入是少数在多工具中表现稳定的优化策略,适合多模型协作场景。

本文通过Oracle到PostgreSQL的迁移任务,研究跨代理规格书的可移植性。实验分两阶段:第一阶段在1,006个PL/SQL文件上评估了基于规格的迁移流程,其中623个成功重构,380个生成脚本可在PostgreSQL 16中直接运行。第二阶段在包含1,802个Oracle脚本及其对应PostgreSQL实现的数据集上,测试了Amazon Kiro、Google Gemini、GitHub Copilot、Claude Code和Cursor等工具的表现。通过Token F1、精确匹配、SQL语法有效性、AST精确匹配、AST平均相似度及即时可运行性进行评估。结果显示,规格书大小无法预测实现质量,跨代理迁移会导致显著的代理依赖性退化。最强失败案例为Gemini直接使用Kiro生成的规格,其Token F1仅为0.035,语法有效性仅2.33%,AST平均相似度为0.015。重写大幅改善了Gemini表现,压缩未带来通用收益,而检索增强注入是唯一在Gemini与Copilot的帕累托前沿均出现的有效策略。结论表明,在异构软件开发工作流中,规格书不应被视为代理无关资产,需显式考虑规格可移植性、代理特异性解释与基于检索的访问机制。

原文摘要 · Abstract (English)

This paper investigates cross-agent specification portability using Oracle-to-PostgreSQL migration as a controlled software transformation task. The study combines two experimental stages. First, a specification-first migration pipeline was evaluated on 1,006 PL/SQL files, of which 623 were successfully regenerated and 380 generated scripts executed successfully in PostgreSQL 16. Second, cross-agent experiments were conducted on a dataset of 1,802 Oracle scripts with corresponding PostgreSQL implementations using Amazon Kiro, Google Gemini, and GitHub Copilot, with Claude Code and Cursor included in the initial single-agent evaluation. Native and foreign specifications were assessed using Token F1, exact match, SQL syntax validity, AST exact match, AST mean similarity, and immediate runnability. The results show that specification size alone does not predict implementation quality and that cross-agent transfer can produce substantial agent-dependent degradation. The strongest replicated case occurred when Gemini directly consumed a Kiro-origin specification, producing a Token F1 of 0.035, SQL syntax validity of 2.33%, and AST mean similarity of 0.015. Rewriting substantially improved Gemini in the tested configuration, compression did not provide a universal benefit, and retrieval-augmented ingestion was the only common strategy represented on the per-agent Pareto frontiers of both Gemini and Copilot. The findings suggest that specifications in heterogeneous SDD workflows should not automatically be treated as agent-neutral artifacts and motivate explicit consideration of specification portability, agent-specific interpretation, and retrieval-based access in multi-agent software engineering.

大模型编程代码迁移多代理协作

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。