倾听用户声音,发现翻译技术评价的深层分歧
Beyond Accuracy: Community Perspectives on Machine Translation

- 分析4类用户在社交平台对机器翻译的讨论
- 发现技术社区与用户群体对质量、效率看法截然不同
- 适合关注AI落地与社会影响的研究者和从业者
尽管机器翻译(MT)取得了显著进展,非人工智能领域的社区对其提出了越来越多的关切,反映出技术进步与真实用户需求之间的明显差距。例如,自然语言处理研究者关注基准性能,而终端用户更关心伦理问题、信任度、可靠性及成本等。我们主张倾听各用户群体的声音,使研究方向聚焦于他们真正关心的问题。为此,我们首次开展大规模分析,研究四类利益相关者(人工智能开发者、专业译员、语言学习者、语言服务提供商)在社交平台上关于机器翻译技术的讨论。我们构建了一个包含79,286条来自Reddit、Facebook、Bluesky和Mastodon从2019年至2025年的帖子与评论的数据集,并分析了这些群体之间的分歧、差异及其原因。总体而言,我们发现各群体经常存在分歧,甚至在翻译质量、效率和可靠性等议题上表现出强烈的情绪对立。这是因为这些群体看待问题的视角不同:人工智能社区将其视为技术和计算问题,而非人工智能(用户)群体则更关注质量细节、省时效果、用户信任以及更广泛的社会议题。
原文摘要 · Abstract (English)
Despite remarkable progress in machine translation (MT), non-AI communities have raised growing concerns about MT systems, suggesting a noticeable gap between technical advancement and the needs of real-world users. For instance, while NLP researchers focus on benchmark performance, end users care about ethical concerns, trust, reliability, costs, and more. We argue that listening to various user communities is essential so that research efforts would be directed towards the problems that the communities care about. To this end, we present a large-scale analysis, for the first time, that investigates what four stakeholder communities (AI developers, professional translators, language learners, and language service providers) post about MT technology on social media. To do so, we construct a dataset of 79,286 posts and comments from Reddit, Facebook, Bluesky, and Mastodon from 2019 to 2025, and analyse where these communities disagree, and how and why. Overall, we find that communities often disagree, and even show strong conflicts due to polarised sentiments on topics such as translation quality, efficiency, and reliability. This is because these communities approach these topics differently: the AI community frames them as technical and computational problems, while non-AI (user) communities care more about quality nuances, time savings, user trust, and broader social issues.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。