用图结构压缩多智能体辩论，小模型也能高精度推理
SMAGDi: Socratic Multi Agent Interaction Graph Distillation for Efficient High Accuracy Reasoning
- 将多智能体辩论转为带正确性标签的交互图
- 60亿参数学生模型保留400亿模型88%准确率
- 适合需要高效高精度推理的部署场景
多智能体系统虽能提升推理准确率，但因频繁跨智能体辩论导致计算成本高。本文提出SMAGDi，一种将五智能体基于Llama的多智能体系统辩论动态压缩为紧凑的苏格拉底式分解-求解器学生模型的蒸馏框架。SMAGDi将辩论过程建模为有向交互图，节点表示带正确性标签的中间推理步骤，边表示连贯性与跨智能体影响。学生模型通过语言建模、图监督、对比推理和嵌入对齐的复合目标训练，兼顾流畅性与结构化推理。在StrategyQA和MMLU上，SMAGDi将400亿参数多智能体系统压缩至60亿参数学生模型，仍保持88%的原始准确率，显著优于MAGDi、标准知识蒸馏及微调基线。结果表明，显式建模交互图与苏格拉底分解，使小模型可继承多智能体辩论的精度优势，同时具备实际部署效率。
原文摘要 · Abstract (English)
Multi-agent systems (MAS) often achieve higher reasoning accuracy than single models, but their reliance on repeated debates across agents makes them computationally expensive. We introduce SMAGDi, a distillation framework that transfers the debate dynamics of a five-agent Llama-based MAS into a compact Socratic decomposer-solver student. SMAGDi represents debate traces as directed interaction graphs, where nodes encode intermediate reasoning steps with correctness labels and edges capture continuity and cross-agent influence. The student is trained with a composite objective combining language modeling, graph-based supervision, contrastive reasoning, and embedding alignment to preserve both fluency and structured reasoning. On StrategyQA and MMLU, SMAGDi compresses a 40B multi-agent system into a 6B student while retaining 88% of its accuracy, substantially outperforming prior distillation methods such as MAGDi, standard KD, and fine-tuned baselines. These results highlight that explicitly modeling interaction graphs and Socratic decomposition enable small models to inherit the accuracy benefits of multi-agent debate while remaining efficient enough for real-world deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。