用市场竞价机制优化模型微调，效果远超传统平台。
Gradients: When Markets Meet Fine-tuning -- A Distributed Approach to Model Optimisation
- 矿工竞标最优超参数，按性能获得奖励
- 相比商用平台平均提升42.1%，部分任务增益达40%
- 适合追求极致性能的开发者和研究者
当前AutoML平台仍有巨大性能潜力未被挖掘。我们在70M至70B参数规模的模型上测试180个微调任务，发现HuggingFace AutoTrain、TogetherAI、Databricks和Google Cloud始终生成次优配置。Gradients基于Bittensor网络构建，通过竞争机制解决该问题：独立矿工竞相寻找最优超参数，收益与模型性能正相关。这种锦标赛模式探索了单一策略无法触及的配置空间。实验显示，Gradients在对抗TogetherAI、Databricks和Google Cloud时实现100%胜率，在与HuggingFace AutoTrain对比中胜出82.8%；平均性能提升达42.1%。检索增强生成任务提升30-40%，特定人物生成的扩散模型改善23.4%。当矿工为奖励竞争时，发展出集中式方法忽略的优化策略。结果表明，带有经济激励的去中心化系统可系统性超越传统AutoML，提示市场机制或为实现更优微调的关键。代码已开源于https://github.com/rayonlabs/G.O.D.
原文摘要 · Abstract (English)
Current AutoML platforms leave substantial performance untapped. Testing 180 fine-tuning tasks across models from 70M to 70B parameters, we found that HuggingFace AutoTrain, TogetherAI, Databricks, and Google Cloud consistently produce suboptimal configurations. Gradients, built on the Bittensor network, attacks this problem through competition. Independent miners race to find optimal hyperparameters, earning rewards proportional to their models' performance. This tournament drives exploration of configuration spaces that single-strategy methods never examine. In our experiments, Gradients achieved a 100\% win rate against TogetherAI, Databricks, and Google Cloud, and beat HuggingFace AutoTrain in 82.8\% of experiments. Mean improvements reached 42.1\% against commercial platforms. Retrieval-augmented generation tasks saw 30-40\% gains; diffusion models improved 23.4\% on person-specific generation. When miners compete for rewards, they develop optimization strategies that centralized approaches overlook. These findings demonstrate that decentralized systems with economic incentives can systematically outperform traditional AutoML, suggesting market dynamics may be key to achieving superior fine-tuning results. Code is available at https://github.com/rayonlabs/G.O.D.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。