arXiv:2606.07524cs.CLcs.AI2026-06

用可解释性特征构建大模型嵌入,实现高效精准的模型对比与识别。

ABLE: Representing and Mapping LLMs via Attribution-Based Large-model Embedding

论文配图:ABLE: Representing and Mapping LLMs via Attribution-Based Large-model Embedding
图 1 · 摘自论文原文
  • 通过梯度归因与词级对齐,捕捉模型对输入的敏感模式。
  • 在239个开源大模型上验证,关系预测等任务表现优于或媲美现有方法。
  • 无需训练,适合模型溯源、安全分析与选型场景。

大型语言模型(LLMs)的爆炸式增长催生了一个异构且文档不全的生态体系,系统化模型比较对于来源审计、安全分析和模型选择日益重要。现有表示方法难以高效应对这一挑战:分析内部参数的方法在架构兼容时有效,但在结构异构下面临可扩展性障碍;依赖外部输出的方法可能混淆行为相似的模型,且在不同分词器下的输出空间难以对齐。为此,我们提出ABLE(基于归因的大模型嵌入),利用可解释性空间构建模型表示。通过分词器无关的词级对齐聚合梯度归因,ABLE捕捉模型特有的输入敏感性模式,而非仅表面输出。理论上,我们在标准可微Transformer模型假设下证明了ABLE诱导出具有有限样本收敛保证的Lipschitz连续参数-嵌入映射。在239个开源大模型上的大量实验表明,该无训练方法在关系预测、模型路由和基准分数预测任务中达到竞争性或更优性能。

原文摘要 · Abstract (English)

The explosive growth of large language models (LLMs) has created a heterogeneous and poorly documented ecosystem, making systematic model comparison increasingly important for provenance auditing, security analysis, and model selection. Existing representation methods struggle to address this setting efficiently. Approaches analyzing internal parameters are powerful when architectures are compatible, but face scalability barriers under structural heterogeneity, while methods relying on external outputs may conflate models with similar behaviors and are difficult to align in richer output spaces across different tokenizers. To bridge this gap, we propose ABLE (Attribution-Based Large-model Embedding), a framework that leverages the interpretability space to construct model representations. By aggregating gradient-based feature attributions via a tokenizer-agnostic word-level alignment, ABLE captures model-specific input-sensitivity patterns rather than only surface-level outputs. Beyond empirical utility, we provide a stability analysis showing that, under standard regularity assumptions for differentiable Transformer-style models, ABLE induces a Lipschitz-continuous parameter-to-embedding map with finite-sample convergence guarantees. Extensive experiments on 239 open-source LLMs demonstrate that our training-free approach achieves competitive or superior performance in relation prediction, model routing, and benchmark score prediction.

大模型表示可解释性模型对比嵌入学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。