Please use this identifier to cite or link to this item:
http://hdl.handle.net/10397/106863
Title: | Speaker-turn aware diarization for speech-based cognitive assessments | Authors: | Xu, SS Ke, X Mak, MW Wong, KH Meng, H Kwok, TCY Gu, J Zhang, J Tao, W Chang, C |
Issue Date: | 2023 | Source: | Frontiers in neuroscience, 2023, v. 17, 1351848 | Abstract: | Introduction: Speaker diarization is an essential preprocessing step for diagnosing cognitive impairments from speech-based Montreal cognitive assessments (MoCA). Methods: This paper proposes three enhancements to the conventional speaker diarization methods for such assessments. The enhancements tackle the challenges of diarizing MoCA recordings on two fronts. First, multi-scale channel interdependence speaker embedding is used as the front-end speaker representation for overcoming the acoustic mismatch caused by far-field microphones. Specifically, a squeeze-and-excitation (SE) unit and channel-dependent attention are added to Res2Net blocks for multi-scale feature aggregation. Second, a sequence comparison approach with a holistic view of the whole conversation is applied to measure the similarity of short speech segments in the conversation, which results in a speaker-turn aware scoring matrix for the subsequent clustering step. Third, to further enhance the diarization performance, we propose incorporating a pairwise similarity measure so that the speaker-turn aware scoring matrix contains both local and global information across the segments. Results: Evaluations on an interactive MoCA dataset show that the proposed enhancements lead to a diarization system that outperforms the conventional x-vector/PLDA systems under language-, age-, and microphone-mismatch scenarios. Discussion: The results also show that the proposed enhancements can help hypothesize the speaker-turn timestamps, making the diarization method amendable to datasets without timestamp information. |
Keywords: | Comprehensive scoring Dementia detection MOCA Speaker diarization Speaker embedding Speaker-turn timestamps |
Publisher: | Frontiers Research Foundation | Journal: | Frontiers in neuroscience | ISSN: | 1662-4548 | EISSN: | 1662-453X | DOI: | 10.3389/fnins.2023.1351848 | Rights: | © 2024 Xu, Ke, Mak, Wong, Meng, Kwok, Gu, Zhang, Tao and Chang. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) (http://creativecommons.org/licenses/by/4.0/). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms. The following publication Xu SS, Ke X, Mak M-W, Wong KH, Meng H, Kwok TCY, Gu J, Zhang J, Tao W and Chang C (2024) Speaker-turn aware diarization for speech-based cognitive assessments. Front. Neurosci. 17:1351848 is available at https://doi.org/10.3389/fnins.2023.1351848. |
Appears in Collections: | Journal/Magazine Article |
Files in This Item:
File | Description | Size | Format | |
---|---|---|---|---|
fnins-17-1351848.pdf | 1.44 MB | Adobe PDF | View/Open |
Page views
6
Citations as of Jun 30, 2024
Downloads
4
Citations as of Jun 30, 2024
![](/image/google_scholar.jpg)
Google ScholarTM
Check
Altmetric
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.