用位置编码空间的局部性+稀疏采样，高效解释闭源大模型输出。
RoPE-LIME: RoPE-Space Locality + Sparse-K Sampling for Efficient LLM Attribution
- 在旋转位置编码空间计算词相似度，提升掩码下的稳定性。
- 稀疏-K采样策略在有限调用下覆盖更多交互关系。
- 相比传统方法减少API调用，适合资源受限的模型可解释性分析。
解释闭源大语言模型输出具有挑战性，因API访问限制无法使用梯度方法，而依赖重生成文本的扰动法成本高且噪声大。本文提出RoPE-LIME，作为gSMILE的开源扩展，将推理与解释解耦：给定一个封闭模型的固定输出，由小型开源代理模型在输入扰动下，基于概率目标（负对数似然和差异目标）计算分词级归因。RoPE-LIME引入两点改进：(i) 在旋转位置编码（RoPE）空间中基于松弛词移动距离计算的局部性核，实现掩码下的稳定相似性；(ii) 稀疏-K采样策略，在有限预算下提升交互覆盖。在HotpotQA（句子特征）和人工标注的MMLU子集（词语特征）上的实验表明，RoPE-LIME生成的归因更具信息量，优于留一法采样，并在显著减少闭源模型API调用的同时超越gSMILE。
原文摘要 · Abstract (English)
Explaining closed-source Large Language Model (LLM) outputs is challenging because API access prevents gradient-based attribution, while perturbation methods are costly and noisy when they depend on regenerated text. We introduce \textbf{Rotary Positional Embedding Linear Local Interpretable Model-agnostic Explanations (RoPE-LIME)}, an open-source extension of gSMILE that decouples reasoning from explanation: given a fixed output from a closed model, a smaller open-source surrogate computes token-level attributions from probability-based objectives (negative log-likelihood and divergence targets) under input perturbations. RoPE-LIME incorporates (i) a locality kernel based on Relaxed Word Mover's Distance computed in \textbf{RoPE embedding space} for stable similarity under masking, and (ii) \textbf{Sparse-$K$} sampling, an efficient perturbation strategy that improves interaction coverage under limited budgets. Experiments on HotpotQA (sentence features) and a hand-labeled MMLU subset (word features) show that RoPE-LIME produces more informative attributions than leave-one-out sampling and improves over gSMILE while substantially reducing closed-model API calls.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。