arXiv:2410.11330cs.LGcs.NE2024-10被引 3

用进化算法优化训练好的模型,无需梯度也能提升性能。

Evolutionary Retrofitting

  • 用进化优化调整已训练模型的超参数,针对实际误差信号
  • 仅需几十到几百个反馈值,显著少于传统方法
  • 支持人类反馈、非可导指标,适合实时动态调整

AfterLearnER 通过进化优化对已训练完成的机器学习模型进行微调,针对选定参数或超参数,在标准验证集子集上优化一个真实、精确且可能不可导的误差信号。该方法在深度感知中的阈值判断、语音重合成的词错误率、Doom游戏中的击杀/生命数、代码翻译中的计算精度或BLEU分数、3D生成对抗网络的图像质量以及基于潜在扩散模型的图像生成用户反馈等不可导信号上表现出高效性。该微调可在训练后进行,也可在推理时根据用户反馈动态执行。其优势包括高度通用性、可处理不可导反馈(如人工评价,无需梯度)、理论支持的低过拟合风险,以及任意时间行为。最重要的是,AfterLearnER仅需几十至几百个标量反馈,远少于多数相关研究中所需的数万量级。

原文摘要 · Abstract (English)

AfterLearnER (After Learning Evolutionary Retrofitting) consists in applying evolutionary optimization to refine fully trained machine learning models by optimizing a set of carefully chosen parameters or hyperparameters of the model, with respect to some actual, exact, and hence possibly non-differentiable error signal, performed on a subset of the standard validation set. The efficiency of AfterLearnER is demonstrated by tackling non-differentiable signals such as threshold-based criteria in depth sensing, the word error rate in speech re-synthesis, the number of kills per life at Doom, computational accuracy or BLEU in code translation, image quality in 3D generative adversarial networks (GANs), and user feedback in image generation via Latent Diffusion Models (LDM). This retrofitting can be done after training, or dynamically at inference time by taking into account the user feedback. The advantages of AfterLearnER are its versatility, the possibility to use non-differentiable feedback, including human evaluations (i.e., no gradient is needed), the limited overfitting supported by a theoretical study, and its anytime behavior. Last but not least, AfterLearnER requires only a small amount of feedback, i.e., a few dozen to a few hundred scalars, compared to the tens of thousands needed in most related published works.

模型优化进化算法非可导优化轻量反馈

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。