打造可定制的LLM微调工具，让研究更透明高效。
torchtune: PyTorch native post-training library
- 基于PyTorch原生设计，模块化结构便于修改和扩展。
- 在多种场景下表现媲美Axolotl和Unsloth，内存效率高。
- 适合追求可复现、快速迭代的研究者使用。
当前大语言模型通常需多阶段训练流程以获得优异下游性能，其中后训练是适配开源权重模型的主要方式。我们提出torchtune，一个专为简化大语言模型后训练生命周期而设计的PyTorch原生库，支持高效的微调、实验与部署工作流。与许多现有微调框架不同，torchtune不以易用性或特定硬件优化为首要目标，而是强调模块化、可修改性和对底层PyTorch组件的直接访问。本文阐述了torchtune的设计理念，并展示其在模型构建器、训练配方及分布式训练栈中的体现。我们在典型后训练设置中评估该库，对比Axolotl和Unsloth等主流框架，结果表明torchtune在多种场景下均具备优异性能与内存效率，同时保持高度灵活性，适用于快速研究迭代。这些成果使torchtune成为可复现的大语言模型后训练研究的理想基础。
原文摘要 · Abstract (English)
Modern LLMs typically require multistage training pipelines to achieve strong downstream performance, with post-training serving as the main interface for adapting open-weight models. We introduce torchtune, a PyTorch-native library designed to streamline the post-training lifecycle of LLMs, enabling efficient fine-tuning, experimentation, and deployment-oriented workflows. Unlike many existing fine-tuning frameworks, which often optimize for ease of use, specialized recipes, or hardware efficiency at the cost of transparency and extensibility, torchtune emphasizes modularity, hackability, and direct access to the underlying PyTorch components. In this paper, we present the design principles behind torchtune, describe how they are reflected in its model builders, training recipes, and distributed training stack, and evaluate the library across representative post-training settings. We compare against popular fine-tuning frameworks, including Axolotl and Unsloth, and show that torchtune provides strong performance and memory efficiency across many settings while remaining flexible enough for rapid research iteration. These results position torchtune as a practical foundation for reproducible LLMs post-training research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。