arXiv:2607.01903cs.AIcs.SE2026-07

首次为大模型应用的提示层和代码层提供统一复杂度评估工具。

Rethinking Complexity Metrics for LLM-Integrated Applications: Beyond Source Code

论文配图:Rethinking Complexity Metrics for LLM-Integrated Applications: Beyond Source Code
图 1 · 摘自论文原文
  • 将提示视为行为规范,用形式化方法量化提示与代码的结构复杂度
  • 筛选出10个有效指标,其中7个聚焦结构多样性而非简单数量
  • 提示层复杂度独立于代码复杂度,对维护工作量有预测能力

大模型集成应用将自然语言提示与程序代码结合,其运行时行为主要源于提示层而非代码本身。现有复杂度度量仅关注代码层面,忽略提示层的行为逻辑。本文提出HECATE,首个同时评估提示层与代码层复杂度的工具。核心是‘提示即规范’——借鉴霍尔逻辑,将每个提示视为预期行为的规范。基于25个复杂度维度构建52个候选指标,通过18个开源仓库中118个组件的版本历史维护活动验证,剔除在控制代码规模后仍不显著的指标。最终仅10个指标通过检验:7个新指标衡量结构多样性,如大模型调用点、内存属性、提示模板等,称为‘结构广度’;三个传统指标中,RFC具类似特征,而Halstead的N和V仅反映规模影响。即使加入最强代码级指标,提示层指标仍显著,证明提示复杂度是独立维度。在6个外部仓库共20个组件上的验证显示,最优两个指标持续预测维护成本,具备泛化能力。

原文摘要 · Abstract (English)

LLM-integrated applications blend natural language prompts with program code, and much of their runtime behavior originates in the prompt layer rather than in the code itself. Existing complexity metrics, however, operate solely at the code level and therefore overlook this behavioral logic entirely. We present HECATE, the first tool designed to assess complexity in both the prompt and code layers of such applications. Central to HECATE is Prompt-as-Specification, a Hoare-logic-inspired formalism that interprets every prompt as a specification of intended behavior. Grounded in 25 complexity dimensions identified across published taxonomies, the tool generates 52 candidate metrics. We assess each metric against 118 components collected from 18 open-source repositories, relying on maintenance activity derived from version history as an empirical proxy for complexity, and discard any metric that loses significance once code size is accounted for. Only ten metrics withstand this test. Seven belong to our newly introduced set; rather than measuring sheer volume, each tallies structurally distinct elements, such as LLM call sites, memory attributes, and prompt templates, an attribute we call structural breadth. Of the three surviving conventional metrics, RFC exhibits a similar breadth-oriented character, while Halstead N and V survive only as a residual effect of size; our top-performing metrics exceed all three. Crucially, the prompt-layer metrics retain significance even when the strongest code-level metric is added as a covariate, establishing prompt complexity as a dimension in its own right. A final validation on 20 components spanning six held-out repositories shows that the two best-performing metrics continue to predict maintenance effort, supporting their generalizability beyond the training set.

大模型应用复杂度度量提示工程软件维护

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。