An auditory saliency pooling-based LSTM model for speech intelligibility classification

Gallardo Antolín, Ascensión; Montero, Juan Manuel

Publication:
An auditory saliency pooling-based LSTM model for speech intelligibility classification

dc.affiliation.dpto	UC3M. Departamento de Teoría de la Señal y Comunicaciones	es
dc.affiliation.grupoinv	UC3M. Grupo de Investigación: Procesado Multimedia	es
dc.contributor.author	Gallardo Antolín, Ascensión
dc.contributor.author	Montero, Juan Manuel
dc.contributor.funder	Ministerio de Economía y Competitividad (España)	es
dc.contributor.funder	Universidad Carlos III de Madrid	es
dc.date.accessioned	2021-11-29T11:13:17Z
dc.date.available	2021-11-29T11:13:17Z
dc.date.issued	2021-09
dc.description	This article belongs to the Section Computer and Engineering Science and Symmetry/Asymmetry.	en
dc.description.abstract	Speech intelligibility is a crucial element in oral communication that can be influenced by multiple elements, such as noise, channel characteristics, or speech disorders. In this paper, we address the task of speech intelligibility classification (SIC) in this last circumstance. Taking our previous works, a SIC system based on an attentional long short-term memory (LSTM) network, as a starting point, we deal with the problem of the inadequate learning of the attention weights due to training data scarcity. For overcoming this issue, the main contribution of this paper is a novel type of weighted pooling (WP) mechanism, called saliency pooling where the WP weights are not automatically learned during the training process of the network, but are obtained from an external source of information, the Kalinli’s auditory saliency model. In this way, it is intended to take advantage of the apparent symmetry between the human auditory attention mechanism and the attentional models integrated into deep learning networks. The developed systems are assessed on the UA-speech dataset that comprises speech uttered by subjects with several dysarthria levels. Results show that all the systems with saliency pooling significantly outperform a reference support vector machine (SVM)-based system and LSTM-based systems with mean pooling and attention pooling, suggesting that Kalinli’s saliency can be successfully incorporated into the LSTM architecture as an external cue for the estimation of the speech intelligibility level.	en
dc.description.sponsorship	The work leading to these results has been supported by the Spanish Ministry of Economy, Industry and Competitiveness through TEC2017-84395-P (MINECO) and TEC2017-84593-C2-1-R (MINECO) projects (AEI/FEDER, UE), and the Universidad Carlos III de Madrid under Strategic Action 2018/00071/001.	en
dc.format.extent	15
dc.identifier.bibliographicCitation	Gallardo-Antolín, A. & Montero, J. M. (2021a). An Auditory Saliency Pooling-Based LSTM Model for Speech Intelligibility Classification. Symmetry, 13(9), 1728.	en
dc.identifier.doi	https://doi.org/10.3390/sym13091728
dc.identifier.issn	2073-8994
dc.identifier.publicationfirstpage	1728
dc.identifier.publicationissue	9
dc.identifier.publicationtitle	Symmetry	en
dc.identifier.publicationvolume	13
dc.identifier.uri	https://hdl.handle.net/10016/33706
dc.identifier.uxxi	AR/0000028624
dc.language.iso	eng
dc.publisher	MDPI
dc.relation.projectID	Gobierno de España. TEC2017-84395-P	es
dc.relation.projectID	Gobierno de España. TEC2017-84593-C2-1-R	es
dc.rights	© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license.	en
dc.rights	Atribución 3.0 España	*
dc.rights.accessRights	open access	en
dc.rights.uri	http://creativecommons.org/licenses/by/3.0/es/	*
dc.subject.eciencia	Telecomunicaciones	es
dc.subject.other	Speech intelligibility	en
dc.subject.other	LSTM	en
dc.subject.other	Weighted pooling	en
dc.subject.other	Attention	en
dc.subject.other	Saliency	en
dc.subject.other	Auditory saliency model	en
dc.title	An auditory saliency pooling-based LSTM model for speech intelligibility classification	en
dc.type	research article	*
dc.type.hasVersion	VoR	*
dspace.entity.type	Publication