arXiv:2512.12976cs.CL2025-12

让作者在写文档时实时标注主观内容,效果远超第三方标注。

Authors Should Label Their Own Documents

  • 作者在创作时即时回答关于观点的提问,实现真实主观信息采集。
  • 模型点击率提升537%,显著优于行业广告基线。
  • 相比传统方法,该方式更高效、成本更低且质量更高。

第三方标注是文本标注的主流方式,但情感与信念等自我中心信息只能通过第三人称代理近似。本文提出作者标注:在文档创作时由作者即时标注相关内容。我们与拥有超20,000用户的商业聊天机器人合作,部署了实时作者标注系统,可识别任务相关查询,生成动态标注问题,并记录作者实时回答。基于作者标注数据,训练并部署在线学习模型用于产品推荐,以最小化对预设主观信念问题的预测误差。实验显示,该模型点击率较同期行业广告基线提升537%。进一步对比三种传统情感分析标注方法,发现作者标注在质量、获取速度和成本上均更优。研究支持现有文献观点:对于自我中心和主观信念,作者标注比第三方标注显著更高质量。为推动学术应用,我们开放了作者标注服务:https://academic.echogroup.ai。

原文摘要 · Abstract (English)

Third-party annotation is the status quo for labeling text, but egocentric information such as sentiment and belief can at best only be approximated by a third-person proxy. We introduce author labeling, an annotation technique where the writer of the document itself annotates the data at the moment of creation. We collaborate with a commercial chatbot with over 20,000 users to deploy an author labeling annotation system. This system identifies task-relevant queries, generates on-the-fly labeling questions, and records authors' answers in real time. We train and deploy an online-learning model architecture for product recommendation with author-labeled data to improve performance. We train our model to minimize the prediction error on questions generated for a set of predetermined subjective beliefs using author-labeled responses. Our model achieves a 537% improvement in click-through rate compared to an industry advertising baseline running concurrently. We then compare the quality and practicality of author labeling to three traditional annotation approaches for sentiment analysis and find author labeling to be higher quality, faster to acquire, and cheaper. These findings reinforce existing literature that annotations, especially for egocentric and subjective beliefs, are significantly higher quality when labeled by the author rather than a third party. To facilitate broader scientific adoption, we release an author labeling service for the research community at https://academic.echogroup.ai.

作者标注主观标注在线学习产品推荐

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。