用多模型协作与自适应专家网络,提升代码生成的推理能力与鲁棒性。
Boost, Disentangle, and Customize: A Robust System2-to-System1 Pipeline for Code Generation
- 通过蒙特卡洛树搜索结合互验证机制挖掘深层推理过程。
- 将异构数据聚类后训练可组合的LoRA专家,适配不同问题实例。
- 适合需要高可靠代码生成的复杂任务场景,如自动化编程。
大型语言模型(LLMs)在系统1任务中表现卓越,但在系统2任务中的推理机制仍不清晰。本文聚焦代码生成这一典型系统2任务,识别出两大挑战:复杂隐藏的推理过程和异构数据分布带来的训练困难。为此提出BDC框架,利用MC-Tree-Of-Agents算法实现多模型互增强(Boosting),通过DisenLora算法对异构数据聚类并训练可组合的LoRA专家,再借助输入感知超网络动态加权专家以生成定制化求解器。该框架整合了反射式剪枝与精炼的蒙特卡洛树搜索,显著提升模型在复杂推理任务中的效果、灵活性与鲁棒性,为系统2到系统1的转化提供新范式。
原文摘要 · Abstract (English)
Large language models (LLMs) have demonstrated remarkable capabilities in various domains, particularly in system 1 tasks, yet the intricacies of their problem-solving mechanisms in system 2 tasks are not sufficiently explored. Recent research on System2-to-System1 methods surge, exploring the System 2 reasoning knowledge via inference-time computation and compressing the explored knowledge into System 1 process. In this paper, we focus on code generation, which is a representative System 2 task, and identify two primary challenges: (1) the complex hidden reasoning processes and (2) the heterogeneous data distributions that complicate the exploration and training of robust LLM solvers. To tackle these issues, we propose a novel BDC framework that explores insightful System 2 knowledge of LLMs using a MC-Tree-Of-Agents algorithm with mutual \textbf{B}oosting, \textbf{D}isentangles the heterogeneous training data for composable LoRA-experts, and obtain \textbf{C}ustomized problem solver for each data instance with an input-aware hypernetwork to weight over the LoRA-experts, offering effectiveness, flexibility, and robustness. This framework leverages multiple LLMs through mutual verification and boosting, integrated into a Monte-Carlo Tree Search process enhanced by reflection-based pruning and refinement. Additionally, we introduce the DisenLora algorithm, which clusters heterogeneous data to fine-tune LLMs into composable Lora experts, enabling the adaptive generation of customized problem solvers through an input-aware hypernetwork. This work lays the groundwork for advancing LLM capabilities in complex reasoning tasks, offering a novel System2-to-System1 solution.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。