模型间无需数据即可传递知识,实现高效跨模型学习。
Model-to-Model Knowledge Transmission (M2KT): A Data-Free Framework for Cross-Model Understanding Transfer
- 通过概念空间传输知识包,不依赖标签数据或教师输出。
- 在大语言模型上实现85%~90%教师性能,数据使用减少98%以上。
- 适合构建无数据依赖的自进化模型系统,推动AI生态协同演进。
现代人工智能系统高度依赖大规模数据进行训练与模型间知识迁移。知识蒸馏、迁移学习和数据蒸馏虽提升了效率,但仍需教师生成样本、输出或梯度。本文提出模型到模型知识传输(M2KT),一种无需数据的神经网络间概念迁移新范式。M2KT使模型能交换封装了结构化概念嵌入、抽象图、推理轨迹和溯源元数据的知识包。与传统蒸馏不同,M2KT在概念空间而非实例空间运作,且转移过程无需标注数据或教师输出。我们定义了概念流形,引入师生隐空间间的对齐映射,并设计复合损失函数,联合约束几何、结构与推理一致性及显式安全约束。进一步给出教师端知识包生成与学生端接收验证的算法流程。在大语言模型符号推理任务上的实验表明,M2KT可达到教师性能的85%至90%,同时相比标准知识蒸馏降低超过98%的数据使用量。本工作为无数据的AI-to-AI知识迁移奠定了理论与实践基础,助力自进化模型生态系统发展。
原文摘要 · Abstract (English)
Modern artificial intelligence systems depend heavily on large datasets for both training and transferring knowledge between models. Knowledge distillation, transfer learning, and dataset distillation have made such transfers more efficient, yet they remain fundamentally data-driven: a teacher must produce examples, logits, or gradients for a student to learn. In this work, we introduce Model-to-Model Knowledge Transmission (M2KT), a novel paradigm for data-free conceptual transfer between neural networks. M2KT enables models to exchange knowledge packets that encapsulate structured concept embeddings, abstraction graphs, reasoning traces, and provenance metadata. Unlike classical distillation, M2KT operates primarily in concept space rather than example space, and it does not require labeled datasets or teacher-generated outputs during transfer. We formalize the notion of concept manifolds, introduce an inter-model alignment mapping between teacher and student latent spaces, and derive a composite loss that enforces geometric, structural, and reasoning consistency together with explicit safety constraints. We further present algorithmic procedures for teacher-side packet generation and student-side ingestion and verification. Experiments on symbolic reasoning with large language models show that M2KT can achieve approximately 85 to 90 percent of teacher performance while reducing data usage by over 98 percent compared to standard knowledge distillation. This work establishes a theoretical and practical foundation for data-free AI-to-AI knowledge transfer and self-improving model ecosystems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。