开源工具库Bergson助力大规模模型数据归因，提升可解释性研究效率。
Bergson: An Open Source Library for Data Attribution
- 支持分布式训练与磁盘存储梯度，适配超大规模语言模型
- 首次开源实现MAGIC、SOURCE、TrackStar三种前沿数据归因方法
- 专为研究人员设计，简化复杂归因任务的工程负担
数据归因是可解释性领域的重要方向，旨在通过训练数据的影响来解释模型行为，适用于调试异常模型表现和优化训练数据集。然而，大规模实施需大量工程投入，且许多前沿技术缺乏开源工具支持。Bergson是一个开源库，旨在通过支持超大规模语言模型和预训练数据集的归因分析，加速该领域进展。其原生支持磁盘存储梯度与多节点分布式训练，并提供一系列研究友好工具。此外，首次公开实现了三种领先的数据归因方法：MAGIC、SOURCE与TrackStar。项目代码已开放于 https://github.com/EleutherAI/bergson。
原文摘要 · Abstract (English)
Data attribution is a promising field in interpretability that aims to explain model behavior through the influence of its training data, with applications including debugging undesirable model behavior and training dataset curation. However, significant engineering effort is required to perform it at scale, and many cutting edge techniques lack open-source tooling and support. Bergson is an open source library that aims to enable faster progress in the field by providing a host of techniques that scale to very large language models and pre-training datasets. The library natively supports on-disk gradient stores and multi-node distributed training, and provides quality of life tools for researchers. Finally, we introduce the first open-source implementations of three leading data attribution methods: MAGIC, SOURCE, and TrackStar. The library is available at https://github.com/EleutherAI/bergson .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。