让大模型通过文字梯度参与联邦学习，突破无数值损失函数的限制。
Can Textual Gradient Work in Federated Learning?
- 提出FedTextGrad框架，用文本梯度代替数值梯度进行客户端更新
- 实验证明调优本地训练步数等参数对效果至关重要
- 引入信息密度原则提升多端文本聚合质量，适合无损反馈场景
近期研究显示，基于大模型的提示优化（如TextGrad）可通过文本实现自动微分，并反向传播文本反馈，适用于无法支持数值梯度或损失计算的实际场景。本文系统探索将文本梯度引入联邦学习（FL）的可行性与挑战。贡献有四：首先，提出新型联邦文本梯度范式FedTextGrad，客户端上传基于文本梯度优化的提示，服务器聚合这些提示；不同于传统数值聚合的FL框架，该设计专为处理文本数据而生，拓展了联邦学习在缺乏明确数值损失函数问题中的应用范围。其次，基于此设计开展大量实验，揭示调优关键参数（如本地训练步数）的重要性。第三，指出FedTextGrad聚合中的核心挑战：如何保留分布式提示更新的关键信息。最后，针对此问题，改进原始版本，通过运用均匀信息密度原则，在大模型总结客户端提示时提供可操作指导。本研究推动了文本梯度在联邦学习中优化大模型的应用，识别关键问题并指明未来方向，开启一个值得深入探索的新研究领域。
原文摘要 · Abstract (English)
Recent studies highlight the promise of LLM-based prompt optimization, especially with TextGrad, which automates differentiation'' via texts and backpropagates textual feedback. This approach facilitates training in various real-world applications that do not support numerical gradient propagation or loss calculation. In this paper, we systematically explore the potential and challenges of incorporating textual gradient into Federated Learning (FL). Our contributions are fourfold. Firstly, we introduce a novel FL paradigm, Federated Textual Gradient (FedTextGrad), that allows clients to upload locally optimized prompts derived from textual gradients, while the server aggregates the received prompts. Unlike traditional FL frameworks, which are designed for numerical aggregation, FedTextGrad is specifically tailored for handling textual data, expanding the applicability of FL to a broader range of problems that lack well-defined numerical loss functions. Secondly, building on this design, we conduct extensive experiments to explore the feasibility of FedTextGrad. Our findings highlight the importance of properly tuning key factors (e.g., local steps) in FL training. Thirdly, we highlight a major challenge in FedTextGrad aggregation: retaining essential information from distributed prompt updates. Last but not least, in response to this issue, we improve the vanilla variant of FedTextGrad by providing actionable guidance to the LLM when summarizing client prompts by leveraging the Uniform Information Density principle. Through this principled study, we enable the adoption of textual gradients in FL for optimizing LLMs, identify important issues, and pinpoint future directions, thereby opening up a new research area that warrants further investigation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。