arXiv:2609.02852cs.LGcs.CR2026-09

大模型语言输出不可信,安全机制需摆脱依赖语言自述的漏洞。

The Implications of Linguistic Illegibility for LLM Security

  • 提出'语言不可读性'概念,指模型内部计算无法通过语言输出准确反映。
  • 指出依赖语言自述的安全监控(如思维链、特征探针)必然存在漏洞。
  • 推荐用污点追踪等不依赖语言的隔离技术构建更可靠的模型沙箱。

大模型以生成自然语言为训练目标,但多项证据表明其外部语言输出与机械提取的语言特征,难以真实反映模型内部计算过程。本文引入‘语言不可读性’一词,泛指模型语言输出或探针结果无法准确表征模型实际思维的情况。我们主张,当模型内部计算基于激活空间的数学运算而非直接语言表达时,语言不可读性不可避免;此时依赖语言自述的安全机制(如思维链监控、宪法式自我批判、基于语言定义的特征向量探针)永远无法完全可靠。因此,模型沙箱必须采用不依赖读取语言状态的隔离技术。我们提出污点追踪是有效沙箱方案:无论模型如何自述,可预先定义系统状态中哪些部分绝不应受模型数据影响。此外,还讨论了鲁棒虚拟化、第三方审计沙箱配置等补充机制,共同构成对语言监控的最低保障,有助于防范前沿模型近期出现的沙箱漏洞攻击。

原文摘要 · Abstract (English)

LLMs are trained to generate natural language. However, various strands of evidence indicate that an LLM's externalized linguistic outputs and mechanistically-extracted linguistic features can be an unreliable lens for understanding internal model computation. We introduce the term ``linguistic illegibility'' to broadly refer to scenarios in which an LLM's externalized or mechanistically-probed language artifacts fail to represent how the model actually thinks. We argue that the specter of linguistic illegibility is unavoidable for LLMs whose internal computations are not directly expressed via language, but rather math over activation spaces (with lossy translations between activation spaces and natural language happening at the bookends). If linguistic illegibility is always possible, then security mechanisms that rely on a model's linguistic self-reporting (e.g., chain-of-thought monitoring, constitutional self-critique, activation probing for linguistically-defined feature vectors) can never be completely sound; the model sandbox will always need isolation techniques whose guarantees do not depend on reading a model's linguistic state at all. We argue that observing a model's outputs using taint tracking is a promising approach for an effective sandbox: regardless of how a model linguistically self-reports, a taint tracking policy can define, a priori, various pieces of system state that should never be influenced by model-produced data. We also discuss several additional sandboxing mechanisms (e.g., robust virtualization, third-party auditing of sandboxing configurations) which collectively provide a critical floor beneath linguistic monitoring, and would have mitigated recent sandbox exploits by frontier models.

大模型安全语言不可读性沙箱隔离污点追踪

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。