用大模型生成合成数据,让算法选择的元学习更准。
LLM-Driven Performance-Space Augmentation for Meta-Learning-Based Algorithm Selection
- 用大语言模型生成目标区域的合成回归数据
- 均匀采样比聚焦边界更有效,提升多项指标
- 适合做算法选择与元学习研究者参考
基于元学习的算法选择依赖于包含元特征和算法性能标签的元数据集。但真实世界数据集数量有限,导致元数据稀疏,制约了元学习器的泛化能力。本文提出利用大语言模型(LLM)生成合成回归数据,以增强元数据集,生成过程引导至低维性能空间中的目标区域。实验采用二维几何设置,以两个基准算法(landmarkers)的交叉验证R²分数为坐标。对比两种策略:均匀采样(分布于整个性能空间)和基于边界的采样(集中在偏好模糊区)。在42个真实UCI回归数据集和730个合成数据上,两种策略均显著优于无增强基线。其中均匀采样表现更优,相对降低17.47%的汉明损失,提升100.41%的子集准确率,且池化留出外R²提升6.09%。结果支持核心观点:算法性能位于低维流形上,通过用户引导的LLM最大化ε-覆盖,可最小化重建偏差,从而改善算法选择的元学习效果。
原文摘要 · Abstract (English)
Meta-learning for algorithm selection relies on a meta-dataset in which each row corresponds to a supervised learning dataset described by meta-features and labelled with a target value that is associated with algorithm choice (typically, some function of algorithm performance). A persistent limitation is that the number of curated real-world datasets is small, resulting in sparse meta-datasets that constrain meta-learner generalisation. In this paper, we address this problem by augmenting the meta-dataset with synthetic regression datasets produced via a large language model (LLM), with generation steered toward target regions of a low-dimensionality performance space. In our experiments, we adopt a two-dimensional geometric setting defined by the cross-validated $R^2$ scores of two anchor algorithms, known as landmarkers. We compare two augmentation strategies: (1) uniform sampling, which distributes synthetic datasets across the performance space; and (2) margin-based sampling, which concentrates them near the decision boundary where landmarker preference is most ambiguous. Across 42 real-world UCI regression datasets and 730 synthetic datasets, both strategies substantially improve meta-learner performance over the unaugmented baseline under regression and multi-label evaluation formulations. However, uniform augmentation consistently outperforms margin-based augmentation, achieving a 17.47% relative reduction in Hamming loss, a 100.41% relative improvement in subset accuracy, and a +6.09% relative gain in pooled out-of-fold $R^2$. These results lead us to postulate a central thesis: the performance of algorithms resides on a low-dimensional performance manifold, whose reconstruction bias may be minimised by user-guided LLMs that seek to maximise uniform $ε$-cover, and consequently, lead to improved meta-learning for algorithm selection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。