动态选择图表示方式，让大模型更准更快回答图问题
Harnessing Adaptive Topology Representations for Zero-Shot Graph Question Answering
- 根据问题特点自动选最优图表示方法
- 在7个算法题和2个下游任务中提升准确率
- 适合需要高效处理图结构的零样本问答场景
大型多模态模型（LMMs）在多样领域的问题回答任务中展现出强大的零样本能力，包括涉及复杂图拓扑结构的图问答（graph QA）。然而，现有方法通常仅使用单一类型的图表示形式（如统一文本描述或固定视觉风格），缺乏对不同模型或任务偏好的考虑，常导致回答错误或过长。为此，我们首先分析了现有图表示形式的特性与不足，设计了一组专为零样本图问答定制的表示形式 $F_{ZS}$。随后提出新指标“图回答效率”（GRE），衡量回答性能与简洁性的平衡。基于此，构建DynamicTRF框架：先创建基于GRE评分的图表示偏好数据集（TRFP），再训练一个路由模块，在推理时为每道题动态选择最佳的$F_{ZS}$表示。在7个域内算法类图问答任务及2个域外下游任务上的大量实验表明，DynamicTRF显著提升了LMMs在零样本图问答中的准确性。
原文摘要 · Abstract (English)
Large Multimodal Models (LMMs) have shown generalized zero-shot capabilities in diverse domain question-answering (QA) tasks, including graph QA that involves complex graph topologies. However, most current approaches use only a single type of graph representation, namely Topology Representation Form (TRF), such as prompt-unified text descriptions or style-fixed visual styles. Those "one-size-fits-all" approaches fail to consider the specific preferences of different models or tasks, often leading to incorrect or overly long responses. To address this, we first analyze the characteristics and weaknesses of existing TRFs, and then design a set of TRFs, denoted by $F_{ZS}$, tailored to zero-shot graph QA. We then introduce a new metric, Graph Response Efficiency (GRE), which measures the balance between the performance and the brevity in graph QA. Built on these, we develop the DynamicTRF framework, which aims to improve both the accuracy and conciseness of graph QA. To be specific, DynamicTRF first creates a TRF Preference (TRFP) dataset that ranks TRFs based on their GRE scores, to probe the question-specific TRF preferences. Then it trains a TRF router on the TRFP dataset, to adaptively assign the best TRF from $F_{ZS}$ for each question during the inference. Extensive experiments across 7 in-domain algorithmic graph QA tasks and 2 out-of-domain downstream tasks show that DynamicTRF significantly enhances the zero-shot graph QA of LMMs in terms of accuracy
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。