![\"Illustration](\"https:\/\/engineering.jhu.edu\/magazine-archive\/wp-content\/uploads\/2023\/05\/carousel-nolanguage.jpeg\")
\nIn 2010, an earthquake devastated Haiti. Relief workers poured in from around the world. Makeshift cell towers were hastily raised to reestablish phone service. Desperate calls for help came from every direction, yet many went unheeded.<\/p>\n
\u201cNot from a lack of relief workers, but because no one spoke Haitian Creole,\u201d recalls Matthew Wiesner, a research scientist at the Center for Language and Speech Processing<\/a> at Johns Hopkins Whiting School of Engineering.\u00a0<\/span><\/p>\n Wiesner first learned of the Haitian example in 2016 while contributing to a U.S. Department of Defense initiative known as Low Resource Languages for Emergent Incidents (LoReLEI), whose aim was to help translate any language in as little as 24 hours after first encountering it.<\/p>\n The urgency of the Haitian example highlights the complex challenges experts at CLSP have grappled with for decades\u2014bringing the power of natural language processing, like that behind Alexa, Siri, and ChatGPT, to the world\u2019s lesser-spoken languages. Depending on whom you ask, there are between 6,000 and 7,000 languages spoken on planet Earth today. As remarkable as that number is, experts estimate that half the world\u2019s people speak just 100 or so languages. That leaves a staggering four billion people speaking any one of the thousands in that long list of rare languages. \u201cThat is a vast underserved population,\u201d says Sanjeev Khudanpur<\/a>, an associate professor of electrical and computer engineering<\/a> who leads the Center for Language and Speech Processing. \u201cMany languages beyond the top 20 or 30 most-spoken languages don\u2019t work with Siri or Alexa. No one is developing chatbots in these languages.\u201d<\/p>\n A fair number of the 3,000 least-spoken languages in the world are so rare that they are in danger of becoming lost forever as the relative handful of speakers dwindles and no one steps in to keep the languages alive. Many Native American languages are endangered. Younger generations aren\u2019t learning them. These languages don\u2019t work on smartphones or proliferate on the internet.\u00a0<\/span><\/p>\n Other languages present yet different challenges. Several thousand languages are rarely written and many lack a standardized writing system. Others lack digital alphabets to preserve them on computers where they can become valuable data sources. Such is the case in South Asia. The Devanagari alphabet exists on most operating systems, but is not as widespread as the Latin alphabet. And even then, less than half of South Asians speak languages written in Devanagari. Collectively, the population speaking languages lacking suitable digital alphabets could number a few billion people, Khudanpur says.<\/p>\n “Meanwhile voice recognition, translation, and natural language processing using AI are changing the world, but only for a select few languages,\u201d Khudanpur says. \u201cIf the language you speak is among the thousands of outliers, AI is not for you.\u201d<\/p>\n \u201cIf a Quechuan kid in Peru can\u2019t surf the web in Quechua, they\u2019ll use Spanish,\u201d says David Yarowsky<\/a>, a professor of computer science<\/a> and a computational linguist on the CLSP faculty. \u201cThis is a rapidly accelerating train.\u201d<\/p>\n The Center for Language and Speech Processing is one of the foremost academic research centers in the world studying computerized language processing, automatic speech transcription, and machine translation. The center conducts research in many relevant areas including acoustic processing, automatic speech recognition, cognitive modeling, computational linguistics, information extraction, machine translation, and text analysis, among others.\u00a0<\/span><\/p>\n Khudanpur\u2019s handiwork is behind the voice recognition algorithm in Amazon\u2019s Echo\u2014better known as Alexa. He imagines a world where computers and smartphones can instantaneously hear or read any language in the world, understand what\u2019s been said, and then translate those words into any other language on Earth instantaneously\u2014an Amharic Alexa, a Chechen ChatGPT. But that dream is far, far off, he says.<\/p>\n Khudanpur and his colleagues at CLSP are applying their myriad skills to extend AI\u2019s reach to those underserved billions. The cause is one of cultural preservation, but The National Science Foundation has taken particular interest in studying these so-called \u201cunderresourced languages\u201d to preserve their cultural histories. Despite the concerted efforts of CLSP and others in academia and in industry to support them, many languages will fall silent, but the work could at the very least preserve them for posterity, the way archeologists, classicists, and anthropologists learn Latin, ancient Greek, hieroglyphs, and Old English to extract meaning from important cultures of the past.\u00a0<\/span><\/p>\n \u201cMany languages beyond the top 20 or 30 most-spoken languages don\u2019t work with Siri or Alexa. No one is developing chatbots in these languages.\u201d\u00a0<\/span><\/p>\n \u201cLoss of a language is loss of knowledge,\u201d Khudanpur says. \u201cThe Eskimo have some 50 words for snow. Perhaps one describes ice too thin to walk on.\u201d\u00a0<\/span><\/p>\n The U.S. Department of Defense considers the study of rare and dying languages as a matter of national security. LoReLEI is but one example. Recall the urgent pleas for speakers of Urdu, Pashto, and tribal languages of Afghanistan and Pakistan in the early 2000’s Afghanistan War. Understanding a rare language might distinguish a recipe for a babka from a recipe for a bomb or unlock a trove of information in a captured laptop. In a public health setting, knowing a rare language might help beat back the next pandemic or speed relief to victims of natural disasters like those victims of the Haitian earthquake whose calls for help went unmet.\u00a0<\/span><\/p>\n \u201cJohns Hopkins is one of the major forces in trying to provide the language technologies that will help make the thousands of languages remain viable,\u201d Yarowsky explains.\u00a0<\/span><\/p>\n Like many at CLSP, Yarowsky uses machine learning to study the structure and rules of language. Increasingly, the challenge comes down to data, or the lack of it\u2014a deficit experts are referencing when they describe a language as \u201cunderresourced.\u201d There aren\u2019t vast stores of written text or hours of recorded speech to work from. Books and other materials that are a rich resource for the more widespread tongues are rarely translated into the lesser-spoken languages.\u00a0<\/span><\/p>\n \u201cThere\u2019s a very long tail of languages that get left behind. And unfortunately, resources like GPT-3 and technologies like ChatGPT, Alexa, and Siri, even the internet itself, are making the digital divide worse,\u201d Yarowsky says.<\/p>\n GPT-3, the large language model behind ChatGPT\u2019s \u201cintelligence,\u201d contains some 250 billion words of written text\u2014almost all in English. That\u2019s more than 900,000 Bibles Yarowsky first got interested in language as a Rockefeller Fellow traveling the Himalayas soon after college. Roaming the villages, living among the Nepalis, learning their languages, he grew fascinated by the way different peoples express similar meaning. That passion became his life\u2019s work. He became a computational linguist and uses mathematics, algorithms, and artificial intelligence to map languages to one another. But mapping of words is just \u201cscaffolding,\u201d as Yarowsky puts it. He is more deeply interested in semantics and the meaning of language.\u00a0<\/span><\/p>\n To illustrate, Yarowsky turns to one of the Bible\u2019s shortest but best-known verses, \u201cDavid killed Goliath.\u201d In his hands, those Hemingway-esque words reveal layers of In a few words, we not only learn that David is \u201cDabide<\/i>\u201d in Japanese, but Goliath is \u201cGoriate<\/i>\u201d and killed is \u201ctaoshita<\/i>\u201d by word-aligning the two languages as in Figure 1. We also learn that in Japanese, the actor (the person who\u2019s doing the killing) is denoted with a \u201cga<\/i>.\u201d \u201cDabide ga<\/i>\u201d is the killer. The subject of the action, the slain giant, is denoted with a \u201cwo<\/i>\u201d\u2014\u201cGoriate wo<\/i>.\u201d And that \u201ctaoshita<\/i>\u201d is a modification of the verb \u201ctaosu<\/i>\u201d\u2014to kill\u2014placing the action in the past tense, Yarowsky says.\u00a0<\/span><\/p>\n Yarowsky is not doing such analyses for Japanese alone, nor for 10, 20, 100, or even 500 languages. \u201cWe\u2019re doing this for all 1,600 Bible translations,\u201d he says. \u201cEssentially, every language that has a written form.\u201d\u00a0<\/span><\/p>\n For Yarowsky, each of the Bible\u2019s many translations is a neatly mapped, word-for-word, sentence-for-sentence, verse-for-verse database upon which to train his algorithms. There are even 24 English variations of the Bible from which to draw meaning.\u00a0<\/span><\/p>\n \u201cIt\u2019s like 1,600 Rosetta Stones,\u201d he says. \u201cEach with hundreds of thousands of words of English and translated word for word in all those languages. In the linguistic context, it’s actually not a lot of data, but it\u2019s enough to do a lot.\u201d\u00a0<\/span><\/p>\n The joke around CLSP is that faculty member Philipp Koehn<\/a> wrote the book on machine translation\u2014literally. In fact, he wrote two. The first was his foundational 2009 textbook Statistical Machine Translation <\/i><\/a>and the second was 2020\u2019s Neural Machine Translation<\/i><\/a>.\u00a0<\/span><\/p>\n Koehn pioneered a now widely adopted approach known as phrase-based translation that looks to translate small groups of words instead of individual words. He is also developer of the open source Moses statistical machine translation system. Statistical models use word and phrase probabilities to achieve translation. They were the dominant approach in machine translation until only the past few years, when neural network- based models ascended.<\/p>\n \u201cThe sea change from statistical to neural models happened almost overnight. Now, neural machine translation is used almost exclusively,\u201d says Koehn, a professor of computer science.<\/p>\n Some of Koehn\u2019s most recent work focuses on a many-to-many translation approach known as multilingual machine translation, which translates among three or more languages at the same time. One of his latest publications explores how lower-resourced languages might benefit. The grouping of similar but lesser-known languages capitalizes on linguistic overlaps and expands resources by sharing data from closely related languages. A machine translation system trained to negotiate between English and two related languages, say Assamese and Bengali, will also improve translation between Assamese and Bengali, Koehn\u2019s research shows.<\/p>\n For Koehn, the challenge is similarly one of finding good data in quantity. He says that it takes as many as 10 million words to build a good model\u2014the more, the better. But the typical novel is just 200,000 words. In that regard, one of Koehn\u2019s go-to resources is recordings and written proceedings of European Parliament in which members speak and submit documents in any of the EU\u2019s 24 native tongues, which are then translated into English, the official language of the EU, and the others. The archives of the European Parliament consist of about 15 million words at present.<\/p>\n \u201cI can imagine a day when we have automatic and instantaneous machine translations between almost any language,\u201d Koehn says.\u00a0<\/span><\/p>\n The greatest challenge with underresourced languages is that often the word pairs between two languages have not been formally established, says Kelly Marchisio<\/a>, a doctoral candidate in Koehn\u2019s group who is finding ways to translate unfamiliar languages sight unseen.<\/p>\n It\u2019s a strategy known as \u201cbilingual lexicon induction,\u201d that uses AI and math to establish word pairs. Given a large amount of English text and a similarly large amount in any another language, Marchisio can calculate word pairs in languages she\u2019s never studied before.<\/p>\n In technical terms, Marchisio uses neural networks to calculate a series of mathematical parameters for each word\u2014she calls groups of these parameters \u201cvectors.\u201d Each word in each language is assigned its own vector. She then organizes the terms mathematically in multidimensional digital space, like big balloons of interconnected words where all words of similar meaning in both languages get grouped near one another: colors, animals, travel, weather, and so forth.<\/p>\n \u201cIf \u2018brown\u2019 and \u2018fox\u2019 are often used in conjunction in English, then they are likely to be similarly combined in German, as well,\u201d Marchisio says. With additional analysis, she can start to pair words from both languages.<\/p>\n On the precipice of earning her doctorate, Marchisio says she likes the democratizing nature of her work and helping to keep languages alive. Ideally, she hopes to bring the power of the internet and artificial intelligence to underserved peoples.<\/p>\n \u201cEverybody deserves access to information,\u201d Marchisio says. \u201cAnd they should be able do it in their native language.\u201d<\/p>\n Matthew Wiesner\u2019s Haitian relief example highlights the broad complexity of language processing. His realm is the spoken word, which introduces wholly new sets of challenges to CLSP. The frustrations experienced in Haiti birthed the Defense Advanced Research Projects Agency\u2019s LoReLEI, an effort to develop a language processing technology that works quickly, in a matter of hours, even with a language never heard before. Such capabilities hold great promise in low-resource language settings.<\/p>\n \u201cInstead of working on Swahili, Zulu, Telugu, or Bengali, all of a sudden, we were working in Ilocano, spoken in the Philippines. Oromo, the second or third language of Ethiopia. Odia, the eighth or ninth language of India,\u201d Wiesner recalls. \u201cOr, as in Haiti, Creole.\u201d<\/p>\n These emergency approaches don\u2019t need to be perfect, Wiesner says, they just need to work, and work quickly. He uses an approach known as cross-language modeling. In this strategy, AI helps to polish recorded speech, removing the unusable artifacts in the audio\u2014background noise, the emotion of the speaker, and so forth.<\/p>\n The focus is strictly on the phonetics. Given a relatively few bits of recorded content, Wiesner builds a phonetic library of all the sounds in a language (the phonemes, in the linguist\u2019s terminology) and assigns them unique characters (the graphemes). This allows him to transcribe longer recorded pieces phonetically.<\/p>\n \u201cIf you give me a recording of 30 minutes of someone speaking a language I\u2019ve never heard before, in less that 24 hours we can create a transcription model that \u201cFor instance, in English we have two \u2018P\u2019 sounds,\u201d Wiesner explains. \u201cThe P sound at the beginning of words, like \u2018please\u2019 where there’s a little H after the P,\u201d he says, emphasizing the pronunciation, \u2018puh-lease.\u2019 \u201cAnd then there\u2019s the P sound after an S as in \u2018spin.\u2019 We need a way to write both.\u201d\u00a0<\/span><\/p>\n With these phonetic transcripts, Wiesner does not care about spelling or even where one word stops and another starts. He just wants accurate and consistent phonetic spellings. Then, as with printed text, those phonetic spellings serve as patterns to which neural networks can apply traditional language processing techniques.\u00a0<\/span><\/p>\n Wiesner has found that these rapid cross-language strategies work best by pooling all graphemes of all languages in a single universal library\u2014one collection of all the spoken sounds of all the world\u2019s languages.\u00a0<\/span><\/p>\n \u201cIf you give me a recording of 30 minutes of someone speaking a language I\u2019ve never heard before, in less that 24 hours, we can create a transcription model that gets half the words right,\u201d Wiesner says. \u201cIt\u2019s not perfect, but it doesn\u2019t need to be to work in an emergency situation where understanding is more important that accuracy.\u201d\u00a0<\/span><\/p>\n Despite these collective efforts at CLSP and tremendous progress across the field in general, language processing remains a complex, evolving landscape with many unanswered questions and areas of opportunity. The challenges are so vast, in fact, that Khudanpur has added the role of convener to his already-considerable duties. Since 1995, the CLSP has been bringing together like-minded researchers and professionals from around the language processing world for the Frederick Jelinek Summer Workshop in Speech and Language Technology<\/a>.\u00a0<\/span><\/p>\n![\"\"](\"https:\/\/engineering.jhu.edu\/magazine-archive\/wp-content\/uploads\/2023\/05\/Screenshot-2023-05-31-at-2.01.59-PM-300x178.png\")
<\/p>\n![\"Mongolian](\"https:\/\/engineering.jhu.edu\/magazine-archive\/wp-content\/uploads\/2023\/05\/Screenshot-2023-05-31-at-2.01.00-PM-300x168.png\")
It is a matter of digital equity, Khudanpur stresses. Today\u2019s ubiquitous voice recognition applications, like Alexa and Siri, and artificially intelligent chatbots, like ChatGPT, are concentrated on a relative handful of the most-spoken languages\u2014English especially. As AI becomes more powerful, the world\u2019s most-common languages will benefit. The rest will fall behind. Too many will die.\u00a0<\/span><\/p>\n![\"Classe](\"https:\/\/engineering.jhu.edu\/magazine-archive\/wp-content\/uploads\/2023\/05\/Screenshot-2023-05-31-at-2.01.15-PM-300x220.png\")
increasingly, Khudanpur says, also a matter of national security, of global public health, and of human rights.<\/p>\n![\"image](\"https:\/\/engineering.jhu.edu\/magazine-archive\/wp-content\/uploads\/2023\/05\/Sanjeev-clipped-copy-2-300x298.jpeg\")
<\/b><\/p>\n
\n\u2014 Sanjeev Khudanpur<\/b><\/p><\/blockquote>\nHOLY WORDS\u00a0<\/span><\/h2>\n
\nworth of written content, which just happens to be Yarowsky\u2019s go-to data source. The Bible has been translated into more than 1,500 languages by missionaries evangelizing in remote corners of the globe.<\/p>\n![\"image](\"https:\/\/engineering.jhu.edu\/magazine-archive\/wp-content\/uploads\/2023\/05\/green-chat-bubble.png\")
<\/span>
\nmeaning. Speaking the verse in Japanese, Yarowsky intones, \u201cDabide ga goriate wo taoshita.<\/i>\u201d\u00a0<\/span><\/p>\nWELCOME TO THE MACHINE\u00a0<\/span><\/h2>\n
![\"diagram](\"https:\/\/engineering.jhu.edu\/magazine-archive\/wp-content\/uploads\/2023\/05\/Scaffolding-Mapping-Word-Pairs.png\")
SHARED VECTORS<\/h2>\n
![\"image](\"https:\/\/engineering.jhu.edu\/magazine-archive\/wp-content\/uploads\/2023\/05\/blue-chat-bubble.png\")
\u201cImagine I talk to you for 10 hours in English about a random topic,\u201d Marchisio says, laying out a hypothetical. \u201cThen I switch to German, which you don\u2019t speak, and talk for another 10 hours on a completely different topic. With just that amount of data, we can write programs that learn to translate between English and German without any interceding examples of translation.\u201d<\/p>\nTRANSLATING AT THE SPEED OF SOUND
![\"illustration](\"https:\/\/engineering.jhu.edu\/magazine-archive\/wp-content\/uploads\/2023\/05\/NLLB-144x300.jpeg\")
<\/h2>\n
\ngets half the words right.\u201d \u2014 MATTHEW WIESNER\u00a0<\/span><\/b><\/p><\/blockquote>\nWORKSHOPPING IT\u00a0<\/span><\/h2>\n
![\"black-and-white](\"https:\/\/engineering.jhu.edu\/magazine-archive\/wp-content\/uploads\/2023\/05\/shutterstock_1247716711_bitmap-copy-300x200.jpeg\")
From the Mouths of Babes<\/h2>\n