评估语言模型代理的身份稳定性，提升其可信与持续能力。
Agent Identity Evals: Measuring Agentic Identity
- 提出一套基于统计的实证框架，衡量代理身份随时间的保持程度。
- 识别并量化代理在状态扰动下的恢复能力与行为一致性。
- 适合研究智能体可靠性、记忆系统设计的开发者与研究人员。
语言模型代理（LMAs）的自主能力与可信度依赖于其能否长期维持稳定可靠的身份。然而，LMAs继承了大语言模型（LLMs）的固有缺陷（如状态无记忆性、随机性、对提示敏感、语言中介依赖），这些会削弱其可识别性、连续性、持久性与一致性，进而影响推理、规划与行动等自主能力。为应对这一挑战，我们提出「代理身份评估」（Agent Identity Evals, AIE），一个严谨、统计驱动的实证框架，用于测量和评估LMA系统在时间维度上维持其代理身份的程度，包括能力、属性及从状态扰动中恢复的能力。AIE包含一组新指标，可与性能、能力及鲁棒性评估集成，辅助优化LMA基础设施与支撑架构（如记忆模块与工具）。本文给出了各阶段生命周期的正式定义与应用方法，并提供具体实例。
原文摘要 · Abstract (English)
Central to agentic capability and trustworthiness of language model agents (LMAs) is the extent they maintain stable, reliable, identity over time. However, LMAs inherit pathologies from large language models (LLMs) (statelessness, stochasticity, sensitivity to prompts and linguistically-intermediation) which can undermine their identifiability, continuity, persistence and consistency. This attrition of identity can erode their reliability, trustworthiness and utility by interfering with their agentic capabilities such as reasoning, planning and action. To address these challenges, we introduce \textit{agent identity evals} (AIE), a rigorous, statistically-driven, empirical framework for measuring the degree to which an LMA system exhibit and maintain their agentic identity over time, including their capabilities, properties and ability to recover from state perturbations. AIE comprises a set of novel metrics which can integrate with other measures of performance, capability and agentic robustness to assist in the design of optimal LMA infrastructure and scaffolding such as memory and tools. We set out formal definitions and methods that can be applied at each stage of the LMA life-cycle, and worked examples of how to apply them.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。