arXiv:2504.17421cs.LGcs.AI2025-04被引 8

大模型与小模型协作,高效适配私有领域任务。

Towards Harnessing the Collaborative Power of Large and Small Models for Domain Tasks

  • 大模型向小模型传递知识,小模型反哺大模型,实现双向协同。
  • 在数据隐私和资源受限下,仍能保持模型安全与推理效率。
  • 适合关注隐私保护与资源优化的AI部署研究者参考。

大型语言模型(LLMs)具备广泛泛化能力,但针对特定领域任务需大量数据和计算资源;小型模型(SMs)则更高效且专精于特定领域,但缺乏通用覆盖。采用大模型与小模型协同的策略,可加速LLMs在私有领域的适应,并释放人工智能新潜力。本文综述了近期在大、小模型协同用于私有领域适配方面的进展与挑战,聚焦跨边界环境中的独特约束:模型归属不同参与方,存在数据隐私、模型安全、完整性及资源限制之间的张力。通过分析模型与数据利益相关方间的信息流动,提出一个统一分类体系,将研究分为三类:自上而下的知识迁移(LM→SM)、自下而上的知识回传(SM→LM)以及跨参与方的推理时协作。基于此框架,剖析跨边界信息交换的核心挑战,包括数据隐私、模型安全与完整性威胁、效率瓶颈,并将其整合为多目标优化问题,指导实际部署。最后,梳理此类混合方法的关键开放问题,展望未来研究方向。本文以边界为中心,提供系统性视角,助力隐私敏感、资源高效的AI落地。

原文摘要 · Abstract (English)

Large language models (LMs) offer broad generalization capabilities but require vast amounts of data and computational resources for domain-specific tasks; small models (SMs), in contrast, are more efficient and tailored to specific domains yet lack general-purpose coverage. Taking a collaborative approach, where large and small models work synergistically, can accelerate the adaptation of LLMs to private domains and unlock new potential in AI. This survey presents a comprehensive overview of recent advances and challenges in harnessing the collaborative power of large and small models for private-domain adaptation. It specifically focuses on the unique constraints of cross-boundary environments, where models belong to distinct parties, and examines the resulting tensions among data privacy, model security, integrity, and resource limitations. By analyzing the information flow between distinct model and data stakeholders, we propose a unified taxonomy that classifies research into three primary directions: downward knowledge transfer (LM to SM), upward knowledge transfer (SM to LM), and inference-time collaboration across parties. Drawing on this taxonomy, we analyze the core challenges inherent to cross-boundary information exchange, including data-privacy, model-security, and integrity threats as well as efficiency constraints, and synthesize these into a multi-objective optimization problem that governs practical deployment. Finally, we review key open challenges inherent to such hybrid approaches and outline promising directions for future research. By offering a principled, boundary-centric view of this rapidly evolving landscape, this survey aims to serve as a structured resource for researchers and practitioners advancing privacy-aware, resource-efficient AI deployment.

大模型小模型协同学习隐私保护

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。