AutoAdapt自动优化大模型领域适配，省去人工调参，提升小数据场景下的准确率。
AutoAdapt: An Automated Domain Adaptation Framework for LLMs
- 采用多智能体辩论机制，自动规划适配策略并融合数据信号与最佳实践
- 引入基于LLM的代理模型，在有限预算下高效优化超参数，提升适配效率
- 在10个任务中平均准确率提升25%，适合资源受限场景下的自动化模型适配
大语言模型在开放领域表现优异，但在数据有限、知识动态变化的特定领域表现不佳。现有领域适配方法依赖大量人工试错，超参数复杂度高，对数据和用户偏好敏感，且训练成本高昂。此外，超参数在不同模型/领域的可迁移性尚不明确，导致适配效果难以预测。为此，我们提出AutoAdapt，一个端到端的自动化大模型领域适配框架。该框架利用文献与开源资源构建的精选知识库，减少专家干预。为缩小搜索空间，设计新型多智能体辩论系统，由提议与批判智能体迭代交互，将用户意图、数据信号与最佳实践融入规划过程。为在严格预算下优化超参数，提出AutoRefine——一种基于LLM的代理模型，替代耗时的黑箱搜索。在10个任务上，AutoAdapt相较当前最优自动化机器学习基线，平均相对准确率提升25%，且开销极低。
原文摘要 · Abstract (English)
Large language models (LLMs) excel in open domains but struggle in specialized settings with limited data and evolving knowledge. Existing domain adaptation practices rely heavily on manual trial-and-error processes, incur significant hyperparameter complexity, and are highly sensitive to data and user preferences, all under the high cost of LLM training. Moreover, the interactions and transferability of hyperparameter choices across models/domains remain poorly understood, making adaptation gains uncertain even with substantial effort. To solve these challenges, we present AutoAdapt, a novel end-to-end automated framework for efficient and reliable LLM domain adaptation. AutoAdapt leverages curated knowledge bases from literature and open-source resources to reduce expert intervention. To narrow the search space, we design a novel multi-agent debating system in which proposal and critic agents iteratively interact to align user intent and incorporate data signals and best practices into the planning process. To optimize hyperparameters under tight budgets, we propose AutoRefine, a novel LLM-based surrogate that replaces costly black-box search. Across 10 tasks, AutoAdapt achieves a 25% average relative accuracy improvement over state-of-the-art Automated Machine Learning baselines with minimal overhead.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。