arXiv:2601.03808cs.CVcs.LG2026-01被引 12

让大模型学会根据性能反馈自动设计数据增强,省去盲目试错。

From Brute Force to Semantic Insight: Performance-Guided Data Transformation Design with LLMs

  • 用实测准确率作为反馈信号,让大模型自主优化数据变换策略。
  • 只需不到1/600的候选方案,就能达到与暴力搜索相当的最高精度。
  • 适合需要高效数据增强的机器学习研究者和工程团队。

大型语言模型在代码生成中表现优异,但数据增强仍受限于启发式设计或暴力搜索。本文提出一种性能感知的闭环方法,在NNGPT项目生态中,使大模型通过内化实证性能信号,自主设计最优变换。基于超过6000个经实测评估的PyTorch增强函数(仅标注下游模型准确率),使用低秩微调(LoRA)训练模型,采用成对性能排序(优-劣)进行对齐,无需强化学习、奖励模型或符号目标。该方法显著减少搜索量,相较暴力发现最多降低600倍候选数量,同时保持竞争力峰值精度,并将生成从随机合成转向任务对齐设计。消融实验表明,结构化思维链提示引入语法噪声并降低性能,而直接提示可确保性能关键任务中的稳定优化。定性和定量分析显示,模型内化的是语义性能线索而非语法记忆。结果表明,大模型可通过非文本反馈环实现任务级推理,绕过显式符号奖励。

原文摘要 · Abstract (English)

Large language models (LLMs) have achieved notable performance in code synthesis; however, data-aware augmentation remains a limiting factor, handled via heuristic design or brute-force approaches. We introduce a performance-aware, closed-loop solution in the NNGPT ecosystem of projects that enables LLMs to autonomously engineer optimal transformations by internalizing empirical performance cues. We fine-tune LLMs with Low-Rank Adaptation on a novel repository of more than 6,000 empirically evaluated PyTorch augmentation functions, each annotated solely by downstream model accuracy. Training uses pairwise performance ordering (better-worse transformations), enabling alignment through empirical feedback without reinforcement learning, reward models, or symbolic objectives. This reduces the need for exhaustive search, achieving up to 600x times fewer evaluated candidates than brute-force discovery while maintaining competitive peak accuracy and shifting generation from random synthesis to task-aligned design. Ablation studies show that structured Chain-of-Thought prompting introduces syntactic noise and degrades performance, whereas direct prompting ensures stable optimization in performance-critical code tasks. Qualitative and quantitative analyses demonstrate that the model internalizes semantic performance cues rather than memorizing syntax. These results show that LLMs can exhibit task-level reasoning through non-textual feedback loops, bypassing explicit symbolic rewards.

大模型数据增强性能优化代码生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。