arXiv:2601.12286cs.CLcs.AI2026-01被引 1

用表示工程方法定位大模型对话上下文,识别跑偏的回应。

Conversational Context Classification: A Representation Engineering Approach

  • 通过表示工程提取大模型内部状态中的上下文子空间
  • 在特定领域上对齐上下文,实现90%以上准确率的上下文检测
  • 适合想理解或监管大模型对话行为的研究者与开发者

大型语言模型(LLM)日益普及,但其生成脱离上下文的回应问题亟需解决。传统异常检测难以直接应用于语义上下文。本文提出结合表示工程(RepE)与单类支持向量机(OCSVM)的方法,从LLM的隐藏状态中识别特定上下文的子空间。通过在符合上下文的样本上训练OCSVM,建立隐空间中的稳健边界。实验使用Llama和Qwen两个开源模型,在特定上下文领域中验证了该方法的有效性,成功定位出与目标上下文强相关的模型层。结果表明,该方法能有效识别上下文一致或偏离的对话线程,有助于提升对大模型行为的理解与解释。

原文摘要 · Abstract (English)

The increasing prevalence of Large Language Models (LLMs) demands effective safeguards for their operation, particularly concerning their tendency to generate out-of-context responses. A key challenge is accurately detecting when LLMs stray from expected conversational norms, manifesting as topic shifts, factual inaccuracies, or outright hallucinations. Traditional anomaly detection struggles to directly apply within contextual semantics. This paper outlines our experiment in exploring the use of Representation Engineering (RepE) and One-Class Support Vector Machine (OCSVM) to identify subspaces within the internal states of LLMs that represent a specific context. By training OCSVM on in-context examples, we establish a robust boundary within the LLM's hidden state latent space. We evaluate out study with two open source LLMs - Llama and Qwen models in specific contextual domain. Our approach entailed identifying the optimal layers within the LLM's internal state subspaces that strongly associates with the context of interest. Our evaluation results showed promising results in identifying the subspace for a specific context. Aside from being useful in detecting in or out of context conversation threads, this research work contributes to the study of better interpreting LLMs.

大模型安全上下文检测表示工程

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。