通过特征几何预测大模型在组合任务中的失败场景
Adversarial Concept Search: Predicting Compositional Errors From Feature Geometry
- 利用模型表征空间中特征的正交性判断组合能力
- 正交编码时模型准确组合，近似编码时则失败
- 无需测试具体输入即可预判高风险案例，适合模型评测与主动学习
人类难以直观判断大模型面临的挑战性场景。为捕捉这些边缘情况，开发者要么设计对人类也难的问题，要么构建庞大的基准集。我们提出一种新方法：利用大模型的表征几何特性来预测其在哪些概念组合上会失败。研究表明，当两个概念的特征编码近似正交时，模型能可靠地进行系统性组合；而当它们的线性编码相近时，会产生干扰，导致组合失败。该方法在需要系统性组合的任务（如玩具编程、多跳推理、多语言事实回忆）中均能可靠预测失败模式，且无需评估具体输入。这一成果为识别高风险样本、构建针对性压力测试提供了可扩展基础，支持实际部署中的主动学习。
原文摘要 · Abstract (English)
Humans cannot always intuit what scenarios are most challenging to LLMs. Hoping to capture challenging edge cases, developers either design problems to be difficult for humans or curate extensive benchmarks. What if we could instead anticipate which scenarios a model will fail on? In this paper, we use an LLM's representational geometry to predict which concept combinations it will fail on. We attribute this compositional failure to interference between salient features. In tasks that require systematic composition - toy programmatic settings, multihop reasoning, multilingual factual recall - we find that when a pair of concepts is encoded near-orthogonally, the model reliably composes them. When their linear encodings are close, producing interference, the model fails to compose them. Our method reliably anticipates failure modes across different compositional tasks, without evaluating specific inputs. These results lay the groundwork to use representational geometry to identify high-risk examples, construct targeted stress tests, and provide a scalable foundation for active learning in real-world deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。