BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Department of Electrical and Computer Engineering - ECPv6.17.0//NONSGML v1.0//EN
CALSCALE:GREGORIAN
METHOD:PUBLISH
X-WR-CALNAME:Department of Electrical and Computer Engineering
X-ORIGINAL-URL:https://engineering.jhu.edu/ece
X-WR-CALDESC:Events for Department of Electrical and Computer Engineering
REFRESH-INTERVAL;VALUE=DURATION:PT1H
X-Robots-Tag:noindex
X-PUBLISHED-TTL:PT1H
BEGIN:VTIMEZONE
TZID:America/New_York
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:20180311T070000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:20181104T060000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:20190310T070000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:20191103T060000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:20200308T070000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:20201101T060000
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTART;TZID=America/New_York:20191014T100000
DTEND;TZID=America/New_York:20191014T113000
DTSTAMP:20210628T204253Z
CREATED:20210628T204253Z
LAST-MODIFIED:20210628T204253Z
UID:553980-1571047200-1571052600@engineering.jhu.edu
SUMMARY:Dissertation Defense: Vimal Manohar
DESCRIPTION:Title: Semi-supervised training for automatic speech recognition.Abstract: State-of-the-art automatic speech recognition (ASR) systems use sequence-level objectives like Connectionist Temporal Classification (CTC) and Lattice-free Maximum Mutual Information (LF-MMI) for training neural network-based acoustic models. These methods are known to be most effective with large size datasets with hundreds or thousands of hours of data. It is difficult to obtain large amounts of supervised data other than in a few major languages like English and Mandarin. It is also difficult to obtain supervised data in a myriad of channel and envirormental conditions. On the other hand\, large amounts ofunsupervised audio can be obtained fairly easily. There are enormous amounts of unsupervised data available in broadcast TV\, call centers and YouTube for many different languages and in many environment conditions. The goal of this research is to discover how to best leverage the available unsupervised data for training acoustic models for ASR.In the first part of this thesis\, we extend the Maximum Mutual Information (MMI) training to the semi-supervised training scenario. We show that maximizing Negative Conditional Entropy (NCE) over lattices from unsupervised data\, along with state-level Minimum Bayes Risk (sMBR) on supervised data\, in a multi-task architecture gives word error rate (WER) improvements  without needing any confidence-based filtering.In the second part of this thesis\, we investigate using lattice-based supervision as numerator graph to incorporate uncertainities in unsupervised data in the LF-MMI training framework. We explore various aspects of creating the numerator graph including splitting lattices for minibatch training\, applying tolerance to frame-level alignments\, pruning beam sizes\, word LM scale and inclusion of pronunciation variants. We show that the WER recovery rate (WRR) of our proposed approach is 5-10% absolute better than that of the baseline of using 1-best transcript as supervision\, and is stable in the 40-60% range even on large-scale setups and multiple different languages.Finally\, we explore transfer learning for the scenario where we have unsupervised data in a mismatched domain. First\, we look at the teacher-student learning approach for cases where parallel data is available in source and target domains. Here\, we train a “student” neural network on the target domain to mimic a “teacher” neural network on the source domain data\, but using sequence-level posteriors instead of the traditional approach of using frame-level posteriors.We show that the proposed approach is very effective to deal with acoustic domain mismatch in multiple scenarios of unsupervised domain adaptation — clean to noisy speech\, 8kHz to 16kHz speech\, close-talk microphone to distant microphone.Second\, we investigate approaches to mitigate language domain mismatch\, and show that a matched language model significantly improves WRR. We finally show that our proposed semi-supervised transfer learning approach works effectively even on large-scale unsupervised datasets with 2000 hours ofaudio in natural and realistic conditions.
URL:https://engineering.jhu.edu/ece/event/dissertation-defense-vimal-manohar/
LOCATION:Shaffer 302\, Shaffer 302
END:VEVENT
END:VCALENDAR