用大模型重写训练数据，让零阶微调更快更稳。
OAT-Rephrase: Optimization-Aware Training Data Rephrasing for Zeroth-Order LLM Fine-Tuning
- 根据零阶优化特性重写训练数据，提升优化稳定性。
- 在5个分类任务中显著缩小与一阶方法的性能差距。
- 适合资源受限场景下需高效微调大模型的研究者。
使用零阶优化（ZO）微调大语言模型（LLM）可节省内存，但因梯度估计噪声导致收敛慢、优化不稳定。本文提出OAT-Rephrase，一种优化感知的训练数据重写策略，利用大模型理解ZO动态（特别是MeZO）后重写训练样本。该方法采用双阶段流程：重写模型和语义判别器，确保重写后的数据保持任务相关性和逻辑一致性。在五个分类任务和三种LLM架构上的实验表明，OAT-Rephrase持续提升MeZO微调性能，常使性能接近甚至追平一阶方法。结果表明，优化感知重写是一种可复用且开销低的零阶微调增强方案。
原文摘要 · Abstract (English)
Fine-tuning large language models (LLMs) using zeroth-order optimization (ZO) offers a memory-efficient alternative to gradient-based methods but suffers from slower convergence and unstable optimization due to noisy gradient estimates. This paper introduces OAT-Rephrase, an Optimization-Aware Training data rephrasing strategy that leverages an LLM to rephrase training instances based on its understanding of the ZO dynamics, specifically MeZO, derived directly from its paper. The approach incorporates a dual-stage pipeline featuring a rewriter LLM and a semantic judge, ensuring all rephrasings retain task relevance and logical consistency. Evaluations across five classification tasks and three LLM architectures demonstrate that OAT-Rephrase consistently improves MeZO fine-tuning performance, often narrowing or eliminating the gap with first-order methods. Our findings suggest that optimization-aware rephrasing serves as a reusable and low-overhead enhancement for zeroth-order tuning regimes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。