{"id":11915,"date":"2018-11-19T13:21:33","date_gmt":"2018-11-19T18:21:33","guid":{"rendered":"https:\/\/engineering.jhu.edu\/magazine-archive\/?p=11915"},"modified":"2018-11-19T13:34:56","modified_gmt":"2018-11-19T18:34:56","slug":"sweet-sensation","status":"publish","type":"post","link":"https:\/\/engineering.jhu.edu\/magazine-archive\/2018\/11\/sweet-sensation\/","title":{"rendered":"Sweet Sensation"},"content":{"rendered":"\"Sweet<\/a>\n

When graduate student Luke Osborn needed to test the fingertip sensors he\u2019d spent years developing for prosthesis wearers, he didn\u2019t have far to look. The ensuing collaboration and results hold big promise for amputees.<\/p>\n

In a Clark Hall lab one day in the fall of 2016, Luke Osborn, MS \u201914, PhD \u201918, attached two tiny beryllium copper probes to the left arm of his fellow graduate student Gy\u00f6rgy L\u00e9vay, MS \u201917<\/a>.<\/p>\n

L\u00e9vay had lost most of that arm\u2014along with his right hand and both feet\u2014six years earlier, when a severe systemic infection turned his extremities necrotic. Now, he and Osborn were both students in the lab of Nitish Thakor<\/a>, a professor of biomedical engineering and one of the world\u2019s foremost innovators in prosthetic devices.<\/p>\n

\"\"<\/a>Osborn\u2019s copper probes were part of an audacious project\u2014one that he and a group of fellow graduate students had been working on for three years. The goal: to give amputees the ability to feel sensations in the fingertips of their prosthetic hands.<\/p>\n

Under Thakor\u2019s guidance, Osborn had meticulously developed fingertip sensors that mimic the architecture of the neurons in human skin. He had tested the sensors on robotic benchtop prosthetic hands for months, training them to respond to various stimuli.<\/p>\n

Now it was finally time for Osborn to test his sensors on a human subject. L\u00e9vay, who conducted his own separate work on the opposite side of the lab, was glad to volunteer.<\/p>\n

In the movie version of this scene, L\u00e9vay might be overcome with emotion as he feels sensations from his fingertips for the first time in six years. The reality was far more imperfect, laborious, and hit or miss, as L\u00e9vay\u2014a prosthetics technology scientist himself\u2014well knew that it would be.<\/p>\n

“My job was just to sit there and tell Luke what I was feeling at any given moment,\u201d L\u00e9vay says. \u201cIn the beginning, most of the sensations were just electrical irritations, kind of like licking a nine-volt battery.\u201d<\/p>\n

But after hundreds of trials and dozens of hours spread over a six-month period, Osborn\u2019s team perfected its equipment and its methods. By the end of the study, L\u00e9vay could reliably tell, without visual cues, whether his prosthetic hand had picked up a smooth object or a sharp one. When his prosthetic hand picked up sharp objects, he felt sensations in his artificial fingertips that seemed like actual tactile pain, not just electrical jolts.<\/p>\n

The study\u2014which was published in Science Robotics<\/em><\/a> last June\u2014marks a breakthrough in providing sensory inputs for prosthesis users. For Osborn, it was the culmination of five years of work.<\/p>\n

\"Luke<\/a>
Luke Osborn, center, and Nitish Thakor, right, in the lab, where testing of the prosthesis began in 2016. (Image: Will Kirk \/ Homewood Photography)<\/figcaption><\/figure>\n

\u201cPart of the beauty of the field of biomedical engineering<\/a>,\u201d he says, \u201cis that it\u2019s getting stronger on both the medical and the engineering sides. This is exactly the kind of work that I hoped to do when I came to Hopkins.\u201d<\/p>\n

Thakor calls Osborn\u2019s project one of the most impressive he has ever supervised. \u201cLuke built the sensors, wrote the algorithms, and designed the experiment,\u201d he says. \u201cHe had the experience and the competence to do all of this, and he put it all toward solving an exciting problem. It came together beautifully in the end.\u201d<\/p>\n

 <\/p>\n

A Hard Nut to Crack<\/strong><\/h2>\n

Osborn arrived in Thakor\u2019s lab in the fall of 2012. As an undergraduate at the University of Arkansas, Osborn had primarily been interested in pure robotics. But by the time he started graduate school, he wanted to do work that had medical applications. Thakor\u2019s lab seemed like the perfect fit. For more than 25 years, Thakor has worked on developing better methods for controlling prosthetic limbs, both at Johns Hopkins<\/a> and at a companion lab in Singapore.<\/p>\n

Osborn quickly turned his attention to the problem of sensation. There have been many improvements in control systems for prosthetic devices during the last decade, but few attempts have been made to allow amputees to feel touch signals from their prosthetic limbs.<\/p>\n

\u201cTouch is really interesting,\u201d says Paul Marasco, a sensory neurophysiologist who works on bionic prosthetic devices at Cleveland Clinic. (He was not involved in Osborn\u2019s project.) \u201cThe individual touch sensors in the skin don’t really provide you with a cohesive perception of touch. The brain has to take all that information and put it all together and make sense of it. The brain says, ‘Well, OK, if I have a little bit of this and a little bit of that, and there\u2019s a little bit of this sprinkled in, then that probably means that this object is slippery. Or this object is made out of metal, or it feels soft.’ None of those perceptions map simply onto a single type of sensory neuron. All of them require the brain to integrate data from several different types of neurons. That\u2019s one reason why sensation has been such a hard nut to crack and why there are so few labs doing it.\u201d<\/p>\n

Osborn was determined to try. He began his project in 2013 by looking for sensor materials that would be flexible enough to fit smoothly over the fingertips of a prosthesis but tough enough to withstand repeated contact with diverse objects. After several rounds of trial and error, he developed a rubber fabric that encases two layers of small silicone-based piezoresistive sensors. The team calls this fabric the \u201ce-dermis.\u201d<\/p>\n

The two layers of the e-dermis mimic the two primary layers of human skin: the epidermis and the dermis. On the outer, \u201cepidermal\u201d layer, Osborn\u2019s team designed certain sensors to behave like human nociceptors, which detect noxious, painful stimuli. In the deeper, \u201cdermal\u201d layer, the sensors mirror four types of neurons known as mechanoreceptors, which variously detect light touch and sustained pressure.<\/p>\n

\u201cIt\u2019s a pattern that\u2019s biomimetic\u2014a sensor array that matches what our nerve endings are used to,\u201d Thakor says. \u201cLuke\u2019s team made a meticulous effort here to get the patterns right.\u201d<\/p>\n

As he developed the fingertip sensors, Osborn initially performed benchtop experiments using a prosthetic hand that was not attached to a human user. In these purely robotic trials, he developed two reflex responses that mimic human spinal reflexes. First, he trained the hand to tighten its grip if the fingertip sensors told it that an object was slipping. Second, he trained the hand to automatically drop a painful object.<\/p>\n

The challenge here was speed: Human spinal reflexes operate within 100 to 200 milliseconds\u2014think of how fast you react to a hot stove\u2014and Osborn\u2019s team wanted to match that rate. To accomplish that, the prosthetic hand had to correctly determine within just 70 milliseconds that it was grasping something painful.<\/p>\n

\u201cWe were able to achieve that quick decision by looking at a few key features of the pressure signal from the e-dermis,\u201d Osborn says. \u201cThese features include information such as where the pressure is located, how large the pressure is, and how quickly the pressure changes.\u201d<\/p>\n

 <\/p>\n

An Hour Here, an Hour There<\/strong><\/h2>\n
\"Gy\u00f6rgy<\/a>
Gy\u00f6rgy L\u00e9vay, at Johns Hopkins on a Fulbright, was happy to serve as a test subject for Luke Osborn\u2019s experiments.<\/figcaption><\/figure>\n

By late 2016, with his benchtop studies complete, Osborn was ready to begin testing the e-dermis on human participants. He turned first to fellow grad student L\u00e9vay, who had arrived at Johns Hopkins in 2015 on a Fulbright scholarship. As a master\u2019s degree student in biomedical engineering, L\u00e9vay worked on pattern recognition systems that give prosthesis users better control over their limbs\u2019 movements. (L\u00e9vay is just one of several prosthetic limb users who have studied in Thakor\u2019s lab over the years.)<\/p>\n

Osborn asked L\u00e9vay if he might be willing to be a test subject for his study of painful stimuli. L\u00e9vay said he was absolutely game\u2014particularly since Osborn wasn\u2019t planning to implant electrodes in L\u00e9vay\u2019s skin, an approach that some labs have used with other prosthesis users.<\/p>\n

L\u00e9vay volunteered dozens of hours of his time\u2014an hour here, an hour there\u2014during the final semester of his own master\u2019s degree program.<\/p>\n

The first step was an extended period of sensory mapping. Osborn needed to discover exactly the right locations to place the probes on L\u00e9vay\u2019s residual limb. At most locations, L\u00e9vay simply felt electrical irritation or stinging on the residual limb itself and didn\u2019t perceive any sensations from his prosthetic hand. But at a few sweet spots, which Osborn discovered through many hours of trial and error, L\u00e9vay\u2019s residual nerves could perceive electrical stimulations from the phantom hand only in the phantom hand itself.<\/p>\n

\"\"<\/a>
For Osborn, five years of work have culminated in artificial fingertips that allow prosthesis users like L\u00e9vay to discern sharp from smooth objects and to feel pain.<\/figcaption><\/figure>\n

This is possible, Osborn explains, because the electrical signals he uses are very gentle. \u201cThe current that we\u2019re using is small enough that it wouldn\u2019t typically be perceived by the surface of the skin at the site of stimulation\u201d\u2014that is, the point where the stimulation is attached to L\u00e9vay\u2019s residual arm, he says. \u201cBut some of the nerves underneath the skin do detect the signal, and they interpret it as a signal from the hands that they\u2019re going to send upstream to the brain.\u201d<\/p>\n

Once the sensory mapping was complete, Osborn\u2019s team was able to start working on the heart of the study. As L\u00e9vay\u2019s prosthetic hand grasped smooth and pointed objects, Osborn adjusted the programming of the system, assessing how and where L\u00e9vay was perceiving pain sensations. (The desk was set up so that L\u00e9vay couldn\u2019t see what his prosthetic hand was doing. He didn\u2019t have any visual cues about whether he was grasping smooth objects or sharp ones.)<\/p>\n

Osborn could adjust three primary variables: frequency, amplitude, and pulse width. The goal was to create a \u201cneuromorphic\u201d signal that mirrors the complexity of our perception systems.<\/p>\n

By the end of the study, L\u00e9vay says, he was able to perceive a wide array of touch sensations in his phantom hand. \u201cSome of them were like someone was pressing on my hand or like a pulsating of blood. Some of them were very interesting stuff.\u201d<\/p>\n

\"\"<\/a>
Osborn and team developed an \u201ce-dermis\u201d for the artificial fingertips, which mimic the two primary layers of human skin.<\/figcaption><\/figure>\n

Over the course of more than 150 trials, Osborn developed a complex algorithm that gave L\u00e9vay a reasonably accurate set of pain perceptions from the prosthetic device. The locations of the pain perceptions were never as pinpoint-specific as an intact person would have experienced\u2014nor were they ever expected to be. But L\u00e9vay could correctly report whether the pain was occurring along the median nerve (the region of the thumb and index finger) or the ulnar nerve (the pinkie). Electroencephalogram studies confirmed that the signals were activating regions of L\u00e9vay\u2019s brain that corresponded to the phantom hand.<\/p>\n

Throughout the project, Osborn checked in with Thakor at weekly research meetings. The team also included Andrei Dragomir, a senior research fellow at the National University of Singapore; Whiting School doctoral students Joseph Betthauser \u201914 and Christopher Hunt \u201914; and Harrison Nguyen \u201918, who helped design and test the final iterations of the fingertip sensors.<\/p>\n

As the youngest member of the team, Nguyen says that he had a valuable experience working with Osborn and Thakor. \u201cDepending on where you are in your training, Luke can be very supportive and hands-on,\u201d he says. \u201cAnd once you\u2019re more capable, he\u2019s glad to be more hands-off. He\u2019s always willing to talk through problems in the lab.\u201d<\/p>\n

 <\/p>\n

Wave of the Future<\/strong><\/h2>\n

\"\"<\/a>You might imagine that the dozens of hours they spent sitting together in the lab would have allowed Osborn and L\u00e9vay to talk shop and to exchange ideas about their mutual interest in improving prosthetic devices. But it wasn\u2019t quite like that: To maintain the integrity of the experiment, it was crucial for L\u00e9vay to be blinded to many of the questions Osborn was trying to answer. When L\u00e9vay described what a stimulus felt like, Osborn wanted his description to be based purely on what he was feeling, not biased by any knowledge of how Osborn was programming the system.<\/p>\n

\u201cLuke went to surprisingly painstaking lengths to make sure I didn\u2019t know what he was up to,\u201d L\u00e9vay says. \u201cFor instance, the stimulator had a little red light on it that blinked every time a stimulation was sent. So if I\u2019d really watched it, I could have deduced the frequency of the stimulation. Luke taped it off so that I couldn\u2019t see it. His screens were always hidden away, and I could only look in a certain direction. So, yeah, it was hard, because I was really interested in what was happening.\u201d<\/p>\n

This was particularly frustrating, L\u00e9vay says, \u201cwhen there were sensations that I liked a lot. I would be like, \u2018What were these?\u2019 and Luke would say, \u2018I can’t tell you.\u2019 This lasted for more than a year while I knew basically nothing about what was happening. Of course, we were working in the same lab, so it was that much more difficult. We made sure that we worked on opposite sides of the lab so that I wouldn\u2019t overhear anything accidentally.\u201d<\/p>\n

\"\"<\/a>
The multilayered e-dermis is made up of conductive and piezoresistive textiles encased in rubber. A dermal layer of two piezoresistive sensing elements is separated from the epidermal layer, which has one piezoresistive sensing element, with a 1-mm layer of silicone rubber. The e-dermis was fabricated to fit over the fingertips of a prosthetic hand.<\/figcaption><\/figure>\n

Since completing their work with L\u00e9vay, Osborn\u2019s team has tested sensory perceptions with several other amputees in the Clark Hall lab. To varying degrees, they have all been able to perceive accurate sensory signals from their phantom limbs. One question going forward will be how much the nature of the initial injury affects prosthetic sensory systems. A person who loses a limb in a military conflict, for example, might have different kinds of nerve damage in the residual limb than a person who loses a limb from septicemia or from a motor vehicle accident.<\/p>\n

\u201cSome of the crucial factors,\u201d L\u00e9vay says, \u201care the level of skin degradation that occurred. Is the skin that remains on the individual sensitive? Is it well-vascularized? Did the nerves grow back into the muscles?\u201d<\/p>\n

Osborn, who completed his PhD last summer, hopes to continue working on prosthetic technologies throughout his career. \u201cLuke\u2019s work on sensory input is absolutely the way of the future,\u201d says Rahul Kaliki, the CEO of Infinite Biomedical Technologies<\/a>, a prosthetics-centered firm that spun off from Thakor\u2019s lab in 1997 in partnership with his former student and co-founder, Ananth Natarajan MSE \u201998. \u201cSensory feedback is one of the crucial things that has been missing from prosthetic limbs.\u201d<\/p>\n

Students in Thakor\u2019s Johns Hopkins lab are working on a wide variety of strategies for improving prosthetic devices. In partnership with Infinite Biomedical Technologies, they are developing high- density electrodes for sensing muscle signals and radio-frequency identification systems that allow prostheses to recognize tagged objects\u2014like the user\u2019s personal coffee cup\u2014and to automatically prepare to grasp them. The lab was recently awarded a major grant from the National Science Foundation<\/a> to develop sensors and algorithms for discrimination of texture and shape.<\/p>\n

\u201cLots of robotics labs have developed sensors in the last few years,\u201d Thakor says. \u201cAnd in their proposals, they always say, \u2018This could have applications for prosthetics.\u2019 But they almost never actually do the work to make the sensors useful for amputees. That\u2019s one reason I\u2019m so pleased with what Luke has done.\u201d<\/p>\n

\"\"<\/a>Osborn, for his part, says he is grateful for the many hours volunteered by L\u00e9vay and the other participants in his studies. \u201cNone of what we do would be possible without the interest, dedication, and willingness of participants to come and work with us,\u201d he says.<\/p>\n

Today, back in his native Hungary, L\u00e9vay works remotely as a research director for Infinite Biomedical Technologies. He is continuing to refine his pattern recognition systems for improving users\u2019 control of prosthetic arms.<\/p>\n

He knows from personal experience how high the stakes are. \u201cFor people who have lost limbs, the expectation is very high,\u201d he says. \u201cIf someone gets a prosthesis, what they want is what they lost. And we\u2019re quite a distance away from that. People are not happy at all with the products we have. But that\u2019s what\u2019s prompting further development and research\u2014and results like what Luke has achieved.\u201d<\/p>\n

\n

Web Extra<\/h3>\n