arXiv:2503.10310cs.SEcs.LG2025-03被引 1

提出语义流概念,揭示ML系统内部运行状态。

Capturing Semantic Flow of ML-based Systems

  • 用激活值和嵌入向量构建内部状态流,补充传统控制流
  • 生成语义流图,捕捉模型内部未显式表达的决策
  • 可适配现有动态分析技术,适合研究模型透明性者

基于机器学习的系统(如深度神经网络或大语言模型)虽能实现高性能视觉、自然语言处理与代码生成,但其内部行为对传统动态分析(如测试)仍高度不透明:现有方法仅关注外部可观测结果,如输入相似性或类别标签变化。本文提出「语义流」概念,旨在捕捉这类系统的内部行为,并为传统动态分析技术提供适配平台。语义流融合控制流与来自系统执行的内部状态,如特定层的神经网络激活值或大语言模型代理在推理步骤中的嵌入向量。由此生成的语义流图,可刻画传统控制流中未显式表示的内部决策过程。本文提出语义流理念,以一个深度神经网络和一个大语言模型代理为例进行演示,并简要阐述其性质及如何用于适配现有动态分析技术于基于机器学习的软件系统。

原文摘要 · Abstract (English)

ML-based systems are software systems that incorporates machine learning components such as Deep Neural Networks (DNNs) or Large Language Models (LLMs). While such systems enable advanced features such as high performance computer vision, natural language processing, and code generation, their internal behaviour remain largely opaque to traditional dynamic analysis such as testing: existing analysis typically concern only what is observable from the outside, such as input similarity or class label changes. We propose semantic flow, a concept designed to capture the internal behaviour of ML-based system and to provide a platform for traditional dynamic analysis techniques to be adapted to. Semantic flow combines the idea of control flow with internal states taken from executions of ML-based systems, such as activation values of a specific layer in a DNN, or embeddings of LLM responses at a specific inference step of LLM agents. The resulting representation, summarised as semantic flow graphs, can capture internal decisions that are not explicitly represented in the traditional control flow of ML-based systems. We propose the idea of semantic flow, introduce two examples using a DNN and an LLM agent, and finally sketch its properties and how it can be used to adapt existing dynamic analysis techniques for use in ML-based software systems.

机器学习语义流模型分析动态分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。