提出带类型标注的联邦工具路由框架,实现跨模型结构的安全协作。
Synapse: Federated Tool Routing via Typed Compendium Artifacts
- 用带模式验证的结构化对象替代原始权重,支持细粒度隐私保护
- 在五种分布上验证了合并稳定性与路由鲁棒性,跨四类大模型仅损失约2点
- 适合无共享数据、模型异构的联邦学习场景,尤其支持不匹配架构间迁移
联邦学习中的协作单元决定了可表达的保障能力。传统扁平单元如权重、提示词或原始样本缺乏类型信息,难以实现隐私保护、冲突消解和跨模型迁移等操作。本文提出有类型联邦制品:经过模式验证的结构化对象,其字段结构使每字段差分隐私、模式感知融合与跨架构迁移成为原生操作而非启发式近似。我们构建了SYNAPSE框架,用于在客户端冻结且异构的大语言模型间进行联邦工具路由,无需共享数据或权重,而这是传统方法无法做到的。该框架支持字段级冲突解决的类型化合并算子,对数值元数据提供形式化差分隐私保证,并在五个分布上实证评估了条件检索失真与路由稳定性,其中一个分布违反收缩假设。单个知识库可在四类大模型(LLaMA 3.18B、LLaMA 3.2-3B、Mistral 7B、GPT 4o)间迁移,仅造成约2个百分点性能损失,这是依赖权重共享的联邦方法无法实现的。
原文摘要 · Abstract (English)
The unit of collaboration in federated learning determines what guarantees are even expressible. Flat units like weights, prompts, raw examples, carry no type signature on which privacy, conflict resolution, or cross-model transfer can dispatch as well-defined operations. We propose typed federated artifacts: schema validated objects whose declared field structure makes per field differential privacy, schema aware merging, and cross architectural transfer first-class operations rather than heuristic approximations. We instantiate this as SYNAPSE, a compendium for federated tool routing across clients with frozen, heterogeneous LLMs and no shared data or weights which is a setting flat units cannot handle without either leaking gradients or discarding structure. The compendium admits a typed merge operator with field wise conflict resolution, a formal DP guarantee on numeric metadata, and conditional retrieval distortion and routing-stability results empirically characterized on five distributions, including one where the contraction premise fails. A single compendium transfers across four LLM families (LLaMA 3.18B,LLaMA 3.2-3B, Mistral 7B, GPT 4o) with approximately 2 pt loss, a capability weight-sharing federation cannot provide without architectural matching.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。