New Era for Robust Speech Recognition: Exploiting Deep Learning

433 Pages·2017·8.827 MB·English

by  Shinji Watanabe, Marc Delcroix, Florian Metze, John R. Hershey (eds.)

Shinji Watanabe
Marc Delcroix
Florian Metze
John R. Hershey Editors
New Era for
Robust Speech
Recognition
Exploiting Deep Learning
New Era for Robust Speech Recognition
Shinji Watanabe • Marc Delcroix (cid:129) Florian Metze (cid:129)
John R. Hershey
Editors
New Era for Robust Speech
Recognition
Exploiting Deep Learning
123
Editors
ShinjiWatanabe MarcDelcroix
MitsubishiElectricResearchLaboratories NTTCommunicationScienceLaboratories
(MERL) NTTCorporation
Cambridge Kyoto,Japan
Massachusetts,USA
FlorianMetze JohnR.Hershey
LanguageTechnologiesInstitute MitsubishiElectricResearchLaboratories
CarnegieMellonUniversity (MERL)
Pittsburgh Cambridge
Pennsylvania,USA Massachusetts,USA
ISBN978-3-319-64679-4 ISBN978-3-319-64680-0 (eBook)
DOI10.1007/978-3-319-64680-0
LibraryofCongressControlNumber:2017955274
©SpringerInternationalPublishingAG2017
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof
thematerialisconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation,
broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionorinformation
storageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilarmethodology
nowknownorhereafterdeveloped.
Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication
doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant
protectivelawsandregulationsandthereforefreeforgeneraluse.
Thepublisher,theauthorsandtheeditorsaresafetoassumethattheadviceandinformationinthisbook
arebelievedtobetrueandaccurateatthedateofpublication.Neitherthepublishernortheauthorsor
theeditorsgiveawarranty,expressorimplied,withrespecttothematerialcontainedhereinorforany
errorsoromissionsthatmayhavebeenmade.Thepublisherremainsneutralwithregardtojurisdictional
claimsinpublishedmapsandinstitutionalaffiliations.
Printedonacid-freepaper
ThisSpringerimprintispublishedbySpringerNature
TheregisteredcompanyisSpringerInternationalPublishingAG
Theregisteredcompanyaddressis:Gewerbestrasse11,6330Cham,Switzerland
Thisbookisdedicatedto thememoryof Yajie
Miao((cid:2) 2016),who tragicallypassedaway
duringthepreparationofthisbook.
Preface
Thefieldofautomaticspeechrecognitionhasevolvedgreatlysincetheintroduction
of deep learning, which began only about 5 years ago. In particular, as more and
moreproductsusingspeechrecognitionarebeingdeployed,thereisacrucialneed
for increased noise robustness, which is well served by deep learning methods.
This book coversthe state of the art in noise robustness for deep-neural-network-
based speech recognition with a focus on applications to distant speech. Some of
the main actors in the areas of front-end and back-end research on noise-robust
speech recognition research gathered in Seattle for the 2015 Jelinek Speech and
Language Summer Workshop. They significantly advanced the state of the art by
tightly integrating these two areas together for the first time. This book compiles
their insights, and presents detailed descriptions of some of the key technologies
inthefield, includingspeechenhancement,neural-network-basednoisereduction,
robust features, acoustic-modeladaptation, training data augmentation,novelnet-
work architectures, and training criteria. The presentationof these technologiesis
augmentedwith descriptions of some of the most important benchmarktools and
datasetsthatareinstrumentalforresearchinthefield,andapresentationofrecent
research activities at some of the leading institutions in the area of noise-robust
speechrecognition.
This book is intended for researchers and practitioners working in the field
of automatic speech recognition with an interest in improving noise robustness.
This bookwill also be of interest to graduatestudents in electrical engineeringor
computerscience,whowillfinditausefulguidetothisfieldofresearch.
Cambridge,MA,USA ShinjiWatanabe
Kyoto,Japan MarcDelcroix
Pittsburgh,PA,USA FlorianMetze
Cambridge,MA,USA JohnR.Hershey
vii
Acknowledgments
Much of the work reported in this book was initiated during the 2015 Jelinek
Memorial Summer Workshop on Speech and Language Technologies (JSALT)
at the University of Washington, Seattle, and was supported by Johns Hopkins
University via NSF Grant No. IIS 1005411, and gifts from Google, Microsoft
Research, Amazon, Mitsubishi Electric, and MERL. We would like to thank
thesesponsorcompaniesandJSALTorganizingcommitteemembers,inparticular
ProfessorLesAtlas at Universityof Washington,ProfessorSanjeev Khudanpurat
JohnsHopkinsUniversity,Professor MariOstendorfatUniversityof Washington,
and Dr. Geoffrey Zweig at Microsoft Research. Thank you for providing us the
opportunitytoworkontheseexcitingresearchtopics.
We also would like to thank Dr. Dong Yu and Dr. Mike Seltzer at Microsoft
Researchfortheirvaluablecommentsontheinitialdesignofthebook.
Finally, the editors would like especially to thank all the authorsfor their hard
workandtheirinvaluablecontributions.
ix
Contents
PartI Introduction
1 Preliminaries................................................................ 3
ShinjiWatanabe,MarcDelcroix,FlorianMetze,andJohnR.Hershey
PartII ApproachestoRobustAutomaticSpeechRecognition
2 MultichannelSpeechEnhancementApproachestoDNN-Based
Far-FieldSpeechRecognition............................................. 21
MarcDelcroix,TakuyaYoshioka,NobutakaIto,AtsunoriOgawa,
Keisuke Kinoshita, Masakiyo Fujimoto, Takuya Higuchi,
ShokoAraki,andTomohiroNakatani
3 MultichannelSpatialClusteringUsingModel-BasedSource
Separation................................................................... 51
MichaelI.MandelandJonP.Barker
4 Discriminative Beamforming with Phase-Aware Neural
NetworksforSpeechEnhancementandRecognition .................. 79
XiongXiao,ShinjiWatanabe,HakanErdogan,MichaelMandel,
LiangLu, JohnR. Hershey,MichaelL. Seltzer,GuoguoChen,
YuZhang,andDongYu
5 RawMultichannelProcessingUsingDeepNeuralNetworks.......... 105
TaraN.Sainath,RonJ.Weiss,KevinW.Wilson,ArunNarayanan,
Michiel Bacchiani, Bo Li, Ehsan Variani, Izhak Shafran,
AndrewSenior,KeanChin,AnanyaMisra,andChanwooKim
6 NovelDeepArchitecturesinSpeechProcessing......................... 135
JohnR.Hershey,JonathanLeRoux,ShinjiWatanabe,ScottWisdom,
ZhuoChen,andYusufIsik
xi
xii Contents
7 DeepRecurrentNetworksforSeparationandRecognition
ofSingle-ChannelSpeechinNonstationaryBackgroundAudio...... 165
Hakan Erdogan, John R. Hershey, Shinji Watanabe,
andJonathanLeRoux
8 RobustFeaturesinDeep-Learning-BasedSpeechRecognition ....... 187
VikramjitMitra,HoracioFranco,RichardM.Stern,JulienvanHout,
LucianaFerrer,MartinGraciarena,Wen Wang,DimitraVergyri,
AbeerAlwan,andJohnH.L.Hansen
9 Adaptation of Deep Neural Network Acoustic Models
forRobustAutomaticSpeechRecognition............................... 219
KheChaiSim,YanminQian,GautamMantena,LahiruSamarakoon,
SouvikKundu,andTianTan
10 TrainingDataAugmentationandDataSelection....................... 245
MartinKarafiát,KarelVeselý,KateˇrinaŽmolíková,MarcDelcroix,
Shinji Watanabe, Lukáš Burget, Jan “Honza” Cˇernocký,
andIgorSzo˝ke
11 AdvancedRecurrentNeuralNetworksforAutomaticSpeech
Recognition.................................................................. 261
YuZhang,DongYu,andGuoguoChen
12 Sequence-DiscriminativeTrainingofNeuralNetworks................ 281
GuoguoChen,YuZhang,andDongYu
13 End-to-EndArchitecturesforSpeechRecognition..................... 299
YajieMiaoandFlorianMetze
PartIII Resources
14 The CHiME Challenges: Robust Speech Recognitionin
EverydayEnvironments ................................................... 327
Jon P. Barker, Ricard Marxer, Emmanuel Vincent,
andShinjiWatanabe
15 The REVERB Challenge: A Benchmark Task
forReverberation-RobustASRTechniques ............................. 345
Keisuke Kinoshita, Marc Delcroix, Sharon Gannot,
Emanuël A.P. Habets, Reinhold Haeb-Umbach,
Walter Kellermann, Volker Leutnant, Roland Maas,
Tomohiro Nakatani, Bhiksha Raj, Armin Sehr,
andTakuyaYoshioka
16 DistantSpeechRecognitionExperimentsUsingtheAMICorpus.... 355
SteveRenalsandPawelSwietojanski
Contents xiii
17 ToolkitsforRobustSpeechProcessing................................... 369
Shinji Watanabe, Takaaki Hori, Yajie Miao, Marc Delcroix,
FlorianMetze,andJohnR.Hershey
PartIV Applications
18 SpeechResearchatGoogletoEnableUniversalSpeechInterfaces... 385
MichielBacchiani,FrançoiseBeaufays,AlexanderGruenstein,
PedroMoreno,JohanSchalkwyk,TrevorStrohman,andHeigaZen
19 Challenges in and Solutions to Deep Learning Network
AcousticModelinginSpeechRecognitionProductsatMicrosoft..... 401
YifanGong,YanHuang,KshitizKumar,JinyuLi,ChaojunLiu,
GuoliYe,ShixiongZhang,YongZhao,andRuiZhao
20 AdvancedASRTechnologiesforMitsubishiElectricSpeech
Applications................................................................. 419
YuukiTachioka,ToshiyukiHanazawa,TomohiroNarita,andJunIshii
Index............................................................................... 431