arXiv:2602.04009cs.LGcs.AI2026-02被引 2

分析生成模型在不同提示下的行为差异,找出导致分歧的关键提示。

PromptSplit: Revealing Prompt-Level Disagreement in Generative Models

  • 用张量积嵌入构建提示与输出联合表示,计算核协方差矩阵。
  • 通过特征值空间差异识别模型在特定提示下的行为分歧方向。
  • 支持大规模模型对比,理论保证近似误差可控,适合模型调试与对齐研究。

提示引导的生成式AI模型在视觉和语言领域迅速发展,能从文本输入生成逼真多样结果。随着模型类型日益增多,训练数据和架构各异,亟需系统方法识别哪些提示引发不同模型行为。本文提出PromptSplit,一种基于核的方法,用于检测和分析生成模型间的提示依赖性分歧。针对每对比较模型,该方法通过提示与图像(或文本)特征的张量积嵌入构建联合表示,并计算对应的核协方差矩阵。利用加权差矩阵的特征空间,定位提示层面的行为差异主方向。为提升可扩展性,采用随机投影近似,将复杂度降至$O(nr^2 + r^3)$(投影维数为$r$)。理论分析表明,该近似估计的特征结构期望偏差被控制在$O(1/r^2)$内。在文本到图像、文本到文本及图像描述等场景的实验显示,PromptSplit能准确识别真实行为差异,并定位相关提示,提供可解释的模型分歧检测工具。

原文摘要 · Abstract (English)

Prompt-guided generative AI models have rapidly expanded across vision and language domains, producing realistic and diverse outputs from textual inputs. The growing variety of such models, trained with different data and architectures, calls for principled methods to identify which types of prompts lead to distinct model behaviors. In this work, we propose PromptSplit, a kernel-based framework for detecting and analyzing prompt-dependent disagreement between generative models. For each compared model pair, PromptSplit constructs a joint prompt--output representation by forming tensor-product embeddings of the prompt and image (or text) features, and then computes the corresponding kernel covariance matrix. We utilize the eigenspace of the weighted difference between these matrices to identify the main directions of behavioral difference across prompts. To ensure scalability, we employ a random-projection approximation that reduces computational complexity to $O(nr^2 + r^3)$ for projection dimension $r$. We further provide a theoretical analysis showing that this approximation yields an eigenstructure estimate whose expected deviation from the full-dimensional result is bounded by $O(1/r^2)$. Experiments across text-to-image, text-to-text, and image-captioning settings demonstrate that PromptSplit accurately detects ground-truth behavioral differences and isolates the prompts responsible, offering an interpretable tool for detecting where generative models disagree.

生成模型提示分析模型对比可解释性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。