用图神经网络预测大模型工作流表现，省去重复调用
GNNs as Predictors of Agentic Workflow Performances
- 将智能工作流建模为计算图，用GNN进行性能预测
- 实验表明GNN能高效准确预测工作流表现
- 适合想自动化优化大模型工作流的研究者
由大型语言模型（LLMs）驱动的智能工作流在处理复杂任务方面取得了显著成功。然而，由于需要频繁调用LLM，实际应用中优化这类工作流成本高且效率低。本文提出将智能工作流视为计算图，并倡导使用图神经网络（GNNs）作为其性能的高效预测器，从而避免重复调用LLM进行评估。为实证支持这一观点，我们构建了FLORA-Bench——一个统一平台，用于基准测试GNN在预测智能工作流性能方面的表现。通过大量实验，我们得出结论：GNN是简单但有效的预测工具。这一发现支持了GNN的新应用场景，并开辟了自动化智能工作流优化的新方向。所有代码、模型和数据均开源于https://github.com/youngsoul0731/Flora-Bench。
原文摘要 · Abstract (English)
Agentic workflows invoked by Large Language Models (LLMs) have achieved remarkable success in handling complex tasks. However, optimizing such workflows is costly and inefficient in real-world applications due to extensive invocations of LLMs. To fill this gap, this position paper formulates agentic workflows as computational graphs and advocates Graph Neural Networks (GNNs) as efficient predictors of agentic workflow performances, avoiding repeated LLM invocations for evaluation. To empirically ground this position, we construct FLORA-Bench, a unified platform for benchmarking GNNs for predicting agentic workflow performances. With extensive experiments, we arrive at the following conclusion: GNNs are simple yet effective predictors. This conclusion supports new applications of GNNs and a novel direction towards automating agentic workflow optimization. All codes, models, and data are available at https://github.com/youngsoul0731/Flora-Bench.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。