arXiv:2409.05659cs.SDcs.MM2024-09被引 4

构建跨场景的音视频说话人分离框架,精准识别名人身份。

Audio-Visual Speaker Diarization: Current Databases, Approaches and Challenges

  • 融合音视频信息,适配电视、会议、日常活动等多场景。
  • 提出明星身份精准匹配方法,解决电视场景中的识别难题。
  • 系统梳理现有数据集与前沿方法,推动领域发展。

当前海量音视频内容催生了对鲁棒自动说话人分离系统的需求,以降低人工标注成本,并为各类应用提供说话人信息支持。本文聚焦音视频跨领域内容的整合,提出一种可适应电视、会议及日常活动等多种场景的鲁棒音视频说话人分离框架。区别于现有系统,该框架特别设计了一种方法,用于在出现名人的电视场景中实现精确的身份归属。此外,本文还系统整理了当前主流方法与可用数据集,为后续研究提供基础支持。

原文摘要 · Abstract (English)

Nowadays, the large amount of audio-visual content available has fostered the need to develop new robust automatic speaker diarization systems to analyse and characterise it. This kind of system helps to reduce the cost of doing this process manually and allows the use of the speaker information for different applications, as a huge quantity of information is present, for example, images of faces, or audio recordings. Therefore, this paper aims to address a critical area in the field of speaker diarization systems, the integration of audio-visual content of different domains. This paper seeks to push beyond current state-of-the-art practices by developing a robust audio-visual speaker diarization framework adaptable to various data domains, including TV scenarios, meetings, and daily activities. Unlike most of the existing audio-visual speaker diarization systems, this framework will also include the proposal of an approach to lead the precise assignment of specific identities in TV scenarios where celebrities appear. In addition, in this work, we have conducted an extensive compilation of the current state-of-the-art approaches and the existing databases for developing audio-visual speaker diarization.

说话人分离音视频融合多模态

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。