复现TikTok算法审计发现其结果短期有效且难重复，警示研究需持续追踪。
Revisiting Algorithmic Audits of TikTok: Poor Reproducibility and Short-term Validity of Findings
- 通过重现实验检验TikTok推荐系统审计方法的可复现性。
- 发现多数审计结果仅在短期内成立，随平台变化迅速失效。
- 强调长期、可复现的审计对监管和研究至关重要。
社交媒体平台正日益依赖算法推荐内容，引发用户陷入信息茧房与接触有害内容的担忧。为此，监管机构与研究者呼吁开展系统性算法审计。审计的可复现性与泛化能力至关重要，以验证结论并追踪算法演变。本文复现了现有TikTok推荐系统的傀儡账号（sockpuppeting）审计工作，发现平台变化、内容演化及研究方法本身均导致复现困难，需大幅调整审计流程。实验表明，一次性审计结果通常仅短期有效，其可复现性与泛化性高度依赖方法设计及平台当前状态。这凸显了开展持续、可复现审计的重要性，以动态把握算法变迁。
原文摘要 · Abstract (English)
Social media platforms are constantly shifting towards algorithmically curated content based on implicit or explicit user feedback. Regulators, as well as researchers, are calling for systematic social media algorithmic audits as this shift leads to enclosing users in filter bubbles and leading them to more problematic content. An important aspect of such audits is the reproducibility and generalisability of their findings, as it allows to draw verifiable conclusions and audit potential changes in algorithms over time. In this work, we study the reproducibility of the existing sockpuppeting audits of TikTok recommender systems, and the generalizability of their findings. In our efforts to reproduce the previous works, we find multiple challenges stemming from social media platform changes and content evolution, but also the research works themselves. These drawbacks limit the audit reproducibility and require an extensive effort altogether with inevitable adjustments to the auditing methodology. Our experiments also reveal that these one-shot audit findings often hold only in the short term, implying that the reproducibility and generalizability of the audits heavily depend on the methodological choices and the state of algorithms and content on the platform. This highlights the importance of reproducible audits that allow us to determine how the situation changes in time.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。