无需标注即可发现并操控模型行为，通过梯度分解实现行为解耦与控制。
Gradient Atoms: Unsupervised Discovery, Attribution and Steering of Model Behaviors via Sparse Decomposition of Training Gradients
- 通过预条件特征空间中的字典学习，将训练梯度分解为稀疏原子。
- 500个原子中高一致性者可识别拒绝、算术等无监督行为，且可调控生成率。
- 无需查询-打分阶段，适合探索未知行为，适用于模型可控性研究。
训练数据归因（TDA）方法旨在识别导致模型行为的训练文档。然而，模型常学习跨多个样本共享的泛化概念，而现有TDA方法为有监督范式：需预先定义查询行为，再对每条训练文档进行评分，成本高且无法发现用户未设想的行为。本文提出梯度原子（Gradient Atoms），一种无监督方法，通过在预条件特征空间中使用字典学习，将每文档的训练梯度分解为稀疏成分（即“原子”）。每个原子捕捉由功能相似文档群组引发的共享更新方向，直接揭示了传统按文档方法所忽略的集体结构。在500个发现的原子中，高一致性的原子可无标签地恢复可解释的任务类型行为——如拒绝、算术计算、是/否分类、常识问答。这些原子还可作为有效的控制向量：施加于权重空间扰动后，显著且可控地改变模型行为（如项目符号列表生成率从33%升至94%，系统性拒绝率从50%降至0%）。该方法无需查询-文档评分阶段，其计算复杂度与感兴趣的行为数量无关。代码已开源：https://github.com/jrosseruk/gradient_atoms。
原文摘要 · Abstract (English)
Training data attribution (TDA) methods ask which training documents are responsible for a model behavior. However, models often learn broad concepts shared across many examples. Moreover, existing TDA methods are supervised -- they require a predefined query behavior, then score every training document against it -- making them both expensive and unable to surface behaviors the user did not think to ask about. We present Gradient Atoms, an unsupervised method that decomposes per-document training gradients into sparse components ("atoms") via dictionary learning in a preconditioned eigenspace. Each atom captures a shared update direction induced by a cluster of functionally similar documents, directly recovering the collective structure that per-document methods do not address. Among 500 discovered atoms, the highest-coherence ones recover interpretable task-type behaviors -- refusal, arithmetic, yes/no classification, trivia QA -- without any behavioral labels. These atoms double as effective steering vectors: applying them as weight-space perturbations produces large, controllable shifts in model behavior (e.g., bulleted-list generation 33% to 94%; systematic refusal 50% to 0%). The method requires no query--document scoring stage, and scales independently of the number of query behaviors of interest. Code is available at https://github.com/jrosseruk/gradient_atoms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。