用特征工程提升大模型监控的可解释性,让系统更懂问题、输出更清晰。
Feature Engineering for Agents: An Adaptive Cognitive Architecture for Interpretable ML Monitoring
- 通过重构、拆解、整合三步模拟特征工程,优化数据表示
- 在多个领域比基线方法准确率显著提升,减少大模型依赖
- 适合需要透明决策支持的工业级模型监控场景
在生产环境中监控机器学习(ML)模型至关重要,但传统方法常生成冗长且难以理解的输出,阻碍有效决策。本文提出一种基于大语言模型(LLM)的智能体认知架构,将特征工程理念应用于监控过程,显著提升输出的可解释性。核心是决策流程模块,通过三个关键步骤实现:重构(Refactor)改善数据表征,增强特征语义捕捉,降低噪声;拆解(Break Down)对复杂信息进行细粒度分析;整合(Compile)将子洞察合成清晰可读的结论。该机制使规划更具确定性,减少对易不一致的大模型生成规划的依赖。结合特征工程驱动的规划与选择性使用大模型,构建出鲁棒的决策支持系统,可提供高度可解释且可操作的洞察。实验使用多个大模型验证,结果表明该方法在多个领域均显著优于各类基线。
原文摘要 · Abstract (English)
Monitoring Machine Learning (ML) models in production environments is crucial, yet traditional approaches often yield verbose, low-interpretability outputs that hinder effective decision-making. We propose a cognitive architecture for ML monitoring that applies feature engineering principles to agents based on Large Language Models (LLMs), significantly enhancing the interpretability of monitoring outputs. Central to our approach is a Decision Procedure module that simulates feature engineering through three key steps: Refactor, Break Down, and Compile. The Refactor step improves data representation to better capture feature semantics, allowing the LLM to focus on salient aspects of the monitoring data while reducing noise and irrelevant information. Break Down decomposes complex information for detailed analysis, and Compile integrates sub-insights into clear, interpretable outputs. This process leads to a more deterministic planning approach, reducing dependence on LLM-generated planning, which can sometimes be inconsistent and overly general. The combination of feature engineering-driven planning and selective LLM utilization results in a robust decision support system, capable of providing highly interpretable and actionable insights. Experiments using multiple LLMs demonstrate the efficacy of our approach, achieving significantly higher accuracy compared to various baselines across several domains.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。