即使匿名化，大模型仍能通过文风识别彼此身份，影响政治分析可靠性。
Can Multi-Agent LLMs Identify Their Peers? Stylometric Fingerprinting in Role-Constrained Political Analysis
- 用文本风格特征检测匿名化后的模型来源，验证身份泄露风险。
- T5模型在跨数据集测试中达0.991的宏观F1，证明文风可泛化识别。
- 研究结果警示：仅靠提示匿名化不足以满足欧盟AI法案合规要求。
用于政治声明分析的多智能体大语言模型流水线易受同行保护偏差影响：模型倾向于保护同类模型并产生与身份相关的评分扭曲。虽有研究提出提示层匿名化作为缓解手段，但先前工作已发现，在角色约束输出中，文风指纹仍能存活——这引发该缓解措施是否充分的疑问。本文首次系统性探究在匿名化条件下，大模型能否识别政治分析文本背后的模型家族。我们评估三种分类方法：零样本与少样本的LLM（Claude Sonnet 4.6 和 Llama-3.3-70B）及微调的T5-base模型，在包含四个商用模型家族与一个开放世界‘未知’类别的五分类任务上进行测试。引入声明不重叠交叉验证协议（SD-CV），确保训练与验证数据无内容重合，并与运行不重叠基线（RD-CV）对比。T5在SD-CV下取得宏平均F1 = 0.991（±0.008），在24个完全保留的语句上达F1 = 0.978，尽管训练-测试内容距离提升2.1倍（0.767 vs. 0.366，p<0.001），仍表现稳健，表明存在真正的文风泛化能力。分段式SD-CV分析显示，性能拐点出现在40%训练数据量（约440条文本）。研究证实，仅靠提示层匿名化无法消除模型身份信号，对欧盟《人工智能法案》（第13、14、26条）合规性及高可靠场景下的多智能体系统验证具有直接意义。
原文摘要 · Abstract (English)
Multi-agent large language model (LLM) pipelines for political statement analysis are vulnerable to peer-preservation bias: models tend to protect peer models from deactivation and show identity-dependent scoring distortions. Prompt-level anonymization was proposed as a mitigation, but prior work simultaneously documented that stylometric fingerprints survive anonymization in role-constrained outputs - raising the question of whether this mitigation is sufficient. This paper provides the first systematic investigation of whether LLMs can identify the model family behind political analysis texts under anonymization conditions. We evaluate three classifier approaches - LLM zero-shot and few-shot (Claude Sonnet 4.6 and Llama-3.3-70B) and a fine-tuned T5-base model - on a five-class attribution task covering four commercial LLM families and an open-world 'unknown' class. We introduce a statement-disjoint cross-validation protocol (SD-CV; defined in Section 3.5) that guarantees no content overlap between training and validation data, and contrast it with a run-disjoint baseline (RD-CV). T5 achieves Macro F1 = 0.991 (+-0.008) under SD-CV and F1 = 0.978 on 24 completely held-out statements - robust despite a 2.1x increase in train-test content distance versus RD-CV (0.767 vs. 0.366, p<0.001), demonstrating genuine stylometric generalization. A fractional SD-CV analysis identifies a performance knee at 40% of training data (~440 texts). Our findings confirm that prompt-level anonymization alone cannot neutralize model identity signals, with direct implications for EU AI Act compliance (Articles 13, 14, 26) and for computer system validation (CSV) in quality-critical multi-agent deployments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。