用几何方法快速高效地微调大模型,提升响应质量与速度。
Efficient Test-Time Finetuning of LLMs via Convex Reconstruction and Gradient Caching
- 通过无投影优化选取相关且多样化的训练样本
- 将分数权重转为整数重复样本,降低计算开销
- 适合追求低延迟高精度的实时推理场景
测试时微调(TTFT)是一种新兴范式,通过检索相关序列、更新模型并评估提示来适应每个输入。但其实际应用受限于速度:检索与微调均需在每次查询中完成,成为主要瓶颈。现有方法在速度与质量间权衡:快速检索常冗余,而更具多样性感知的筛选则带来高昂的每查询成本。我们提出HullFT,一种基于几何的TTFT方法。给定查询后,首先利用无投影弗兰克-沃尔夫优化,将查询嵌入表示为少数训练序列的稀疏凸组合,生成天然相关且多样的支持集。随后通过几何整数化过程,将分数凸权重转化为精确整数多重集用于微调。产生的重复样本可借助梯度复用机制,分摊多次微调的前向-反向计算开销。实验表明,相较于当前最先进方法,HullFT在质量-效率权衡上表现更优,以显著更低的总运行时间实现更低的比特/字节值。
原文摘要 · Abstract (English)
Test-time finetuning (TTFT) is a rapidly evolving paradigm that adapts a language model to each prompt by retrieving related sequences, updating the model on them, and then evaluating the prompt. However, TTFT is only practical if it is fast: selection and finetuning both happen per query, making each a direct bottleneck. Existing methods trade speed for quality: fast retrieval is often redundant, while stronger diversity-aware selection adds prohibitive per-query cost. We introduce HullFT, a geometric approach to TTFT that addresses both bottlenecks. Given a query, HullFT first represents the query embedding as a sparse convex combination of few training sequences, using efficient projection-free Frank-Wolfe optimization. This yields a support set that is inherently relevant and diverse. We then convert the fractional convex weights into an exact integer multiset for finetuning through a geometric integerization procedure. The resulting multiplicities naturally create repeated examples, which we exploit with Gradient Reuse to amortize forward-backward computation across repeated finetuning steps. Our experiments show that HullFT improves the quality-efficiency tradeoff over current state-of-the-art TTFT methods, achieving lower bits-per-byte at substantially lower total runtime.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。