用大模型清理加密货币举报数据中的垃圾信息,提升识别准确率。
Clean Up the Mess: Addressing Data Pollution in Cryptocurrency Abuse Reporting Services
- 提出无监督大模型分类器自动识别有效举报和欺诈类型
- 发现75%举报为垃圾信息,0.1%良性地址占60%资金流入
- 可量化诈骗真实收入,投资骗局影响最大,勒索邮件传播广
加密货币举报服务是获取恶意地址、欺诈类型及其对受害者财务影响的重要数据来源,但因其众包特性易受数据污染。本文分析了污染的范围与影响,收集了两年内两个主流平台提交的28.9万条举报记录,回答三个研究问题。RQ1:分析污染程度,发现垃圾举报将最终淹没未受控的服务,如BitcoinAbuse在停运前收到75%的垃圾举报;构建包含19,443条标注报告的公开数据集,揭示用户上报的欺诈类型存在偏差。发现91个(0.1%)正常地址被误报,却涉及60%的资金流入。RQ2:探究自动化识别有效举报及分类的可行性,提出一种无监督大模型分类器,在报告分类上达到F1=0.95,对分布外数据分类得F1=0.89,识别垃圾信息达F1=0.99,显著优于两种基线方法。RQ3:展示该分类器在量化不同欺诈类型财务影响上的实用性,发现受害者报告损失仅低估犯罪收益的1/29,投资骗局造成最高损失,而勒索因大规模邮件推广弥补转化率低的问题。
原文摘要 · Abstract (English)
Cryptocurrency abuse reporting services are a valuable data source about abusive blockchain addresses, prevalent types of cryptocurrency abuse, and their financial impact on victims. However, they may suffer data pollution due to their crowd-sourced nature. This work analyzes the extent and impact of data pollution in cryptocurrency abuse reporting services and proposes a novel LLM-based defense to address the pollution. We collect 289K abuse reports submitted over 6 years to two popular services and use them to answer three research questions. RQ1 analyzes the extent and impact of pollution. We show that spam reports will eventually flood unchecked abuse reporting services, with BitcoinAbuse receiving 75% of spam before stopping operations. We build a public dataset of 19,443 abuse reports labeled with 19 popular abuse types and use it to reveal the inaccuracy of user-reported abuse types. We identified 91 (0.1%) benign addresses reported, responsible for 60% of all the received funds. RQ2 examines whether we can automate identifying valid reports and their classification into abuse types. We propose an unsupervised LLM-based classifier that achieves an F1 score of 0.95 when classifying reports, an F1 of 0.89 when classifying out-of-distribution data, and an F1 of 0.99 when identifying spam reports. Our unsupervised LLM-based classifier clearly outperforms two baselines: a supervised classifier and a naive usage of the LLM. Finally, RQ3 demonstrates the usefulness of our LLM-based classifier for quantifying the financial impact of different cryptocurrency abuse types. We show that victim-reported losses heavily underestimate cybercriminal revenue by estimating a 29 times higher revenue from deposit transactions. We identified that investment scams have the highest financial impact and that extortions have lower conversion rates but compensate for them with massive email campaigns.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。