为AI浏览器插件设计自动化测试框架,解决其非确定性与安全漏洞问题。
ASSURE: Metamorphic Testing for AI-powered Browser Extensions
- 构建模块化测试生成引擎,支持扩展测试场景
- 识别出531个问题,平均12.4分钟发现关键漏洞
- 适合开发团队集成到持续测试流程中
将大语言模型(LLMs)集成到浏览器扩展中,已显著提升网页浏览体验,实现内容摘要、智能翻译和上下文写作辅助等复杂功能。然而,这类AI驱动的扩展带来前所未有的测试与可靠性挑战:传统测试方法无法应对其非确定性行为、上下文敏感性及复杂的网络环境集成。现有LLM测试方法也脱离浏览器具体上下文,形成评估空白。为此,我们提出ASSURE——一个专为AI驱动浏览器扩展设计的模块化自动化测试框架。该框架包含三部分:(1) 支持插件式扩展的测试用例生成引擎;(2) 自动执行框架,协调网页内容、扩展处理与AI模型行为间的复杂交互;(3) 可配置的验证管道,通过行为一致性与安全不变量检测替代精确输出匹配。在六个主流AI浏览器扩展上的评估表明,ASSURE成功识别出531个问题,涵盖安全漏洞、元测试关系违反和内容对齐问题。相比手动测试,测试吞吐量提升6.4倍,平均仅需12.4分钟即可发现关键安全漏洞,具备快速集成至开发流水线的实际可行性。
原文摘要 · Abstract (English)
The integration of Large Language Models (LLMs) into browser extensions has revolutionized web browsing, enabling sophisticated functionalities like content summarization, intelligent translation, and context-aware writing assistance. However, these AI-powered extensions introduce unprecedented challenges in testing and reliability assurance. Traditional browser extension testing approaches fail to address the non-deterministic behavior, context-sensitivity, and complex web environment integration inherent to LLM-powered extensions. Similarly, existing LLM testing methodologies operate in isolation from browser-specific contexts, creating a critical gap in effective evaluation frameworks. To bridge this gap, we present ASSURE, a modular automated testing framework specifically designed for AI-powered browser extensions. ASSURE comprises three principal components: (1) a modular test case generation engine that supports plugin-based extension of testing scenarios, (2) an automated execution framework that orchestrates the complex interactions between web content, extension processing, and AI model behavior, and (3) a configurable validation pipeline that systematically evaluates behavioral consistency and security invariants rather than relying on exact output matching. Our evaluation across six widely-used AI browser extensions demonstrates ASSURE's effectiveness, identifying 531 distinct issues spanning security vulnerabilities, metamorphic relation violations, and content alignment problems. ASSURE achieves 6.4x improved testing throughput compared to manual approaches, detecting critical security vulnerabilities within 12.4 minutes on average. This efficiency makes ASSURE practical for integration into development pipelines, offering a comprehensive solution to the unique challenges of testing AI-powered browser extensions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。