用AI代理检测云配置漂移，即使工具出错也能保持高可靠性。
RIVA: Leveraging LLM Agents for Reliable Configuration Drift Detection
- 设计双代理协作机制，通过交叉验证提升检测鲁棒性。
- 在错误工具输出下准确率从27.3%提升至50.0%，无错误时达43.8%。
- 适合需要高可靠性的生产环境云运维团队使用。
基础设施即代码（IaC）工具可自动化云资源部署，但验证已部署系统是否与IaC规范一致仍具挑战性。配置漂移常由IaC缺陷、手动修改或系统更新引发。基于大语言模型（LLM）的智能体可分析海量遥测数据，适用于配置漂移检测。然而现有系统隐含假设：调用工具总返回正确结果，导致其易受错误响应影响。因智能体无法区分异常输出是真实问题还是工具故障，可能引发漏报或误报，降低本应最可靠的检测能力。我们提出RIVA（Robust Infrastructure by Verification Agents），一种新型多智能体系统，在工具输出错误或误导时仍能实现可靠的IaC验证。RIVA采用验证代理与工具生成代理，通过迭代交叉验证、多视角验证及工具调用历史追踪协作。在AIOpsLab基准测试中，当存在错误工具响应时，RIVA将任务准确率从基线ReAct智能体的27.3%恢复至平均50.0%；在无错误情况下，准确率提升28%至43.8%。结果表明，多样工具调用的交叉验证可显著增强生产环境中自治基础设施验证的可靠性。
原文摘要 · Abstract (English)
Infrastructure as code (IaC) tools automate cloud provisioning but verifying that deployed systems remain consistent with the IaC specifications remains challenging. Such configuration drift occurs because of bugs in the IaC specification, manual changes, or system updates. Large language model (LLM)-based agentic AI systems can automate the analysis of large volumes of telemetry data, making them suitable for the detection of configuration drift. However, existing agentic systems implicitly assume that the tools they invoke always return correct outputs, making them vulnerable to erroneous tool responses. Since agents cannot distinguish whether an anomalous tool output reflects a real infrastructure problem or a broken tool, such errors may cause missed drift or false alarms, reducing reliability precisely when it is most needed. We introduce RIVA (Robust Infrastructure by Verification Agents), a novel multi-agent system that performs robust IaC verification even when tools produce incorrect or misleading outputs. RIVA employs two specialized agents, a verifier agent and a tool generation agent, that collaborate through iterative cross-validation, multi-perspective verification, and tool call history tracking. Evaluation on the AIOpsLab benchmark demonstrates that RIVA, in the presence of erroneous tool responses, recovers task accuracy from 27.3% when using a baseline ReAct agent to 50.0% on average. RIVA also improves task accuracy 28% to 43.8% without erroneous tool responses. Our results show that cross-validation of diverse tool calls enables more reliable autonomous infrastructure verification in production cloud environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。