发现并利用模型错误特征,让冻结预测器自动纠错。
When Do Corrective Features Help? An Agent for Corrective Feature Discovery on Black-Box Forecasters

- 通过分析模型残差,用组合搜索和大模型生成纠错特征。
- 在6个数据集上使最弱模型误差降低27%,效果是纯修正器的两倍。
- 不依赖特定模型,适合想低成本提升现有预测系统的用户。
冻结的预训练预测器常以结构化、重复性方式出错,微调修复成本高。本文研究纠错特征发现:从冻结预测器的残差中挖掘可解释特征,驱动轻量级事后修正器。与传统建模数据生成过程不同,纠错特征聚焦于模型失败机制。提出CRAFTER(基于特征时序探索与推理的纠错残差代理),保持主干冻结,通过两种互补生成器挖掘残差:对原始输入通道进行组合搜索,以及使用大语言模型(LLM)提出命名特征组合、二值标志和简短可执行代码。单个基于验证的门控机制对所有候选特征进行接受或拒绝,无论来源;经验证选择的修正器应用通过的特征,或保持原预测不变。该无源依赖流程亦可用于公平评估已有特征工程系统,使CRAFTER成为仅归因于特征来源改进的工具。在六个公开数据集与六种冻结主干上,CRAFTER在所有特征预算下均优于每个专用特征工程系统,大致将修正器单独带来的提升翻倍,并使最弱主干的误差降低最多达27%。这些收益在不同LLM后端下稳定,且在微调主干上依然有效。
原文摘要 · Abstract (English)
Frozen pretrained forecasters often fail in structured, recurring ways that are costly to repair through fine-tuning. We study corrective feature discovery: mining interpretable features of a frozen forecaster's residual to drive a lightweight post-hoc corrector. Prior automated feature engineering models the data-generating process; corrective features instead model the model-failure process. We present CRAFTER (Corrective Residual Agent with Feature-based Temporal Exploration and Reasoning), which keeps the backbone frozen and mines its residual with two complementary generators: a compositional search over the raw input channels, and a large language model (LLM) that proposes named feature combinations, binary flags, and short executable code. A single validation-grounded gate accepts or rejects every candidate regardless of its origin, and a validation-selected corrector applies the accepted features or leaves the forecast unchanged. This source-agnostic pipeline also allows prior feature-engineering systems to be evaluated under identical conditions, making CRAFTER an instrument for attributing forecast improvements to the feature source alone. Across six public datasets and six frozen backbones, CRAFTER surpasses every dedicated feature-engineering system at every feature budget, roughly doubling the improvement achieved by the corrector alone and reducing the error of the weakest backbones by up to 27%. These gains are robust across different LLM backends and persist even when applied on top of fine-tuned backbones.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。