用视觉识别自动发现网页欺骗性设计,准确率达93%
Automatically Detecting Online Deceptive Patterns
- 通过截图分析+大模型理解上下文,无需代码即可定位欺骗性元素
- 在真实网页上检测欺骗行为,F1分数达0.93,效果显著
- 适合用户、开发者和监管者使用,支持浏览器插件等三类应用
数字界面中的欺骗性设计会利用认知偏见和心理弱点,诱导用户做出非预期决策,已在各类数字平台广泛存在。尽管法律和技术层面已有应对措施,但可用且可扩展的解决方案仍存在显著缺口。本文提出AutoBot框架,通过截图分析实现对网页欺骗性模式的精准识别与定位,不依赖底层HTML代码。该框架采用两阶段流程:首先利用专用视觉模型分析截图,识别交互元素并提取文本特征;随后借助大语言模型理解上下文,判断是否存在欺骗性模式。我们还使用AutoBot构建合成数据集,将大模型知识蒸馏至小型语言模型。大量实验表明,AutoBot在网页上检测欺骗性模式时取得0.93的F1得分,展现出强大潜力。我们将其应用于三个下游场景:(1)本地浏览器扩展,为用户提供实时反馈;(2)Lighthouse审计工具,帮助开发者识别网站潜在欺骗设计;(3)面向研究者与监管者的测量工具。
原文摘要 · Abstract (English)
Deceptive patterns in digital interfaces manipulate users into making unintended decisions, exploiting cognitive biases and psychological vulnerabilities. These patterns have become ubiquitous on various digital platforms. While efforts to mitigate deceptive patterns have emerged from legal and technical perspectives, a significant gap remains in creating usable and scalable solutions. We introduce our AutoBot framework to address this gap and help web stakeholders navigate and mitigate online deceptive patterns. AutoBot accurately identifies and localizes deceptive patterns from a screenshot of a website without relying on the underlying HTML code. AutoBot employs a two-stage pipeline that leverages the capabilities of specialized vision models to analyze website screenshots, identify interactive elements, and extract textual features. Next, using a large language model, AutoBot understands the context surrounding these elements to determine the presence of deceptive patterns. We also use AutoBot, to create a synthetic dataset to distill knowledge from 'teacher' LLMs to smaller language models. Through extensive evaluation, we demonstrate AutoBot's effectiveness in detecting deceptive patterns on the web, achieving an F1-score of 0.93 when detecting deceptive patterns, underscoring its potential as an essential tool for mitigating online deceptive patterns. We implement AutoBot, across three downstream applications targeting different web stakeholders: (1) a local browser extension providing users with real-time feedback, (2) a Lighthouse audit to inform developers of potential deceptive patterns on their sites, and (3) as a measurement tool designed for researchers and regulators.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。