BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Department of Electrical and Computer Engineering - ECPv6.17.0//NONSGML v1.0//EN
CALSCALE:GREGORIAN
METHOD:PUBLISH
X-WR-CALNAME:Department of Electrical and Computer Engineering
X-ORIGINAL-URL:https://engineering.jhu.edu/ece
X-WR-CALDESC:Events for Department of Electrical and Computer Engineering
REFRESH-INTERVAL;VALUE=DURATION:PT1H
X-Robots-Tag:noindex
X-PUBLISHED-TTL:PT1H
BEGIN:VTIMEZONE
TZID:America/New_York
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:20190310T070000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:20191103T060000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:20200308T070000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:20201101T060000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:20210314T070000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:20211107T060000
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTART;TZID=America/New_York:20201023T140000
DTEND;TZID=America/New_York:20201023T140000
DTSTAMP:20210628T204259Z
CREATED:20210628T204259Z
LAST-MODIFIED:20210628T204259Z
UID:554220-1603461600-1603461600@engineering.jhu.edu
SUMMARY:Dissertation Defense: Ruizhi Li
DESCRIPTION:Note: This is a virtual presentation. Here is the link for where the presentation will be taking place.Title: An Efficient and Robust Multi-Stream Framework for End-to-End Speech RecognitionAbstract: In the voice-enabled domestic or meeting environments\, distributed microphone arrays aim to process distant-speech interaction into text with high accuracy. However\, with dynamic corruption of noises and reverberations or human movement present\, there is no guarantee that any microphone array (stream) is constantly informative. In these cases\, an appropriate strategy to dynamically fuse streams or select the most informative array is necessary.The multi-stream paradigm in Automatic Speech Recognition (ASR) considers scenarios where parallel streams carry diverse or complementary task-related knowledge. Such streams could be defined as microphone arrays\, frequency bands\, various modalities or etc. Hence\, a robust stream fusion is crucial to emphasize on more informative streams than corrupted ones\, specially under unseen conditions. This thesis focuses on improving the performance and robustness of speech recognition in multi-stream scenarios.In recent years\, with the increasing use of Deep Neural Networks (DNNs) in ASR\, End-to-End (E2E) approaches\, which directly transcribe human speech into text\, have received greater attention. In this thesis\, a multi-stream framework is presented based on joint Connectionist Temporal Classification/Attention (CTC/ATT) E2E model\, where parallel streams are represented by separate encoders. On top of the regular attention networks\, a secondary stream-fusion network is to steer the decoder toward the most informative streams. Two representative frameworks are proposed\, which are Multi-Encoder Multi-Array (MEM-Array) and Multi-Encoder Multi-Resolution (MEM-Res)\, respectively.The MEM-Array model aims at improving the far-field ASR robustness using multiple microphone arrays which are activated by separate encoders. With an increasing number of streams (encoders) requiring substantial memory and massive amounts of parallel data\, a practical two-stage training strategy is desgnated to address these issues. Furthermore\, a two-stage augmentation scheme is present to improve the robustness of the multi-stream model\, where small amount of parallel data is sufficient to achieve competitive results. In MEM-Res\, two heterogeneous encoders with different architectures\, temporal resolutions and separate CTC networks work in parallel to extract complementary information from same acoustics. Compared with the best single-stream performance\, both models have achieved substantial improvement\, which also outperform various conventional fusion strategies.While proposed framework optimizes information in multi-stream scenarios\, this thesis also studies the Performance Monitoring (PM) measures to predict if recognition result of an end-to-end model is reliable\, without growth-truth knowledge. Four different PM techniques are investigated\, suggesting that PM measures on attention distributions and decoder posteriors are well-correlated with true performances.Committee MembersHynek Hermansky\, Department of Electrical and Computer EngineeringShinji Watanabe\, Department of Electrical and Computer EngineeringNajim Dehak\, Department of Electrical and Computer EngineeringGregory Sell\, JHU Human Language Technology Center of Excellence
URL:https://engineering.jhu.edu/ece/event/dissertation-defense-ruizhi-li/
END:VEVENT
END:VCALENDAR