{"id":571571,"date":"2024-12-03T15:21:26","date_gmt":"2024-12-03T20:21:26","guid":{"rendered":"https:\/\/engineering.jhu.edu\/ece\/?post_type=news&#038;p=571571"},"modified":"2024-12-03T15:43:47","modified_gmt":"2024-12-03T20:43:47","slug":"beyond-assistants-how-ai-could-enable-surgical-robots-to-think-and-act-autonomously","status":"publish","type":"news","link":"https:\/\/engineering.jhu.edu\/ece\/news\/beyond-assistants-how-ai-could-enable-surgical-robots-to-think-and-act-autonomously\/","title":{"rendered":"Beyond Assistants: How AI Could Enable Surgical Robots to Think and Act Autonomously"},"content":{"rendered":"<p style=\"font-weight: 400;\">Surgical robots have long been valuable tools in the operating room, assisting surgeons with precision tasks like suturing, holding instruments, or stabilizing organs. However, these systems still depend heavily on direct human control or pre-programmed instructions.<\/p>\n<p style=\"font-weight: 400;\">In a perspective piece in <a href=\"https:\/\/www.nature.com\/articles\/s42256-024-00917-4\"><em>Nature Machine Intelligence<\/em><\/a>, a team including Johns Hopkins engineers proposed using multi-modal, general-purpose models\u2014trained on diverse datasets, including video demonstrations\u2014to make surgical robots safer and more efficient. They say this will equip surgical robots with the adaptability and autonomy needed to handle complex, dynamic procedures, reducing the burden on human surgeons.<\/p>\n<p style=\"font-weight: 400;\">\u201cOne major barrier to achieving fully autonomous surgical robots is the complexity of surgeries themselves,\u201d said lead author Samuel Schmidgall, a graduate research assistant in the Whiting School of Engineering\u2019s <a href=\"https:\/\/engineering.jhu.edu\/ece\/\">Department of Electrical and Computer Engineering<\/a>. \u201cSurgeons work with soft, living tissue, which changes unpredictably. This makes it hard to simulate real-life conditions in training for surgical robots, and they must operate with high levels of safety to avoid patient harm.\u201d And unlike robots in manufacturing, surgical robots cannot rely solely on trial and error; each step must be meticulously planned to avoid risks.<\/p>\n<p style=\"font-weight: 400;\">A single \u201cmulti-modal, multi-task\u201d AI training model that combines visual, language, and physical (action-based) data overcomes these challenges, they say.<\/p>\n<p style=\"font-weight: 400;\">\u201cThis vision-language-action model would allow surgical robots to understand instructions in natural language, interpret what they see in real time, and take appropriate action based on context,\u201d said Schmidgall. \u201cThe model could even detect situations that require human intervention, automatically signaling the surgeon when something is unclear.\u201d<\/p>\n<p style=\"font-weight: 400;\">Axel Krieger, assistant professor in the Whiting School of Engineering\u2019s\u00a0<a href=\"https:\/\/me.jhu.edu\/\">Department of Mechanical Engineering<\/a> and co-author of the paper, recently demonstrated the potential of this approach by <a href=\"https:\/\/hub.jhu.edu\/2024\/11\/11\/surgery-robots-trained-with-videos\/\">training a surgical robot using video-based datasets<\/a>. Krieger\u2019s work showcases how multi-modal training enables robots to perform intricate tasks autonomously, bridging a gap between theory and practice in surgical robotics.<\/p>\n<p style=\"font-weight: 400;\">The emerging multi-modal model used in the Krieger team\u2019s work and discussed in the paper describe the approach\u2019s transformative potential for surgical robotics. By analyzing thousands of procedures, these systems could refine techniques, assist in multiple surgeries simultaneously, and adapt to new scenarios, the authors say, pointing out that such tools could also ease surgeons\u2019 workloads and, ultimately, improve patient outcomes.<\/p>\n<p style=\"font-weight: 400;\">Co-authors of the paper also include Ji Woong Kim from the Department of Mechanical Engineering, Johns Hopkins University; Alan Kuntz from the University of Utah; and Ahmed Ezzat Ghazi, Department of Urology, Johns Hopkins Medical Institute.<\/p>\n","protected":false},"template":"","class_list":["post-571571","news","type-news","status-publish","hentry","news_categories-research"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.7 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Beyond Assistants: How AI Could Enable Surgical Robots to Think and Act Autonomously - Department of Electrical and Computer Engineering<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/engineering.jhu.edu\/ece\/news\/beyond-assistants-how-ai-could-enable-surgical-robots-to-think-and-act-autonomously\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Beyond Assistants: How AI Could Enable Surgical Robots to Think and Act Autonomously - Department of Electrical and Computer Engineering\" \/>\n<meta property=\"og:description\" content=\"Surgical robots have long been valuable tools in the operating room, assisting surgeons with precision tasks like suturing, holding instruments, or stabilizing organs. However, these systems still depend heavily on&hellip;\" \/>\n<meta property=\"og:url\" content=\"https:\/\/engineering.jhu.edu\/ece\/news\/beyond-assistants-how-ai-could-enable-surgical-robots-to-think-and-act-autonomously\/\" \/>\n<meta property=\"og:site_name\" content=\"Department of Electrical and Computer Engineering\" \/>\n<meta property=\"article:modified_time\" content=\"2024-12-03T20:43:47+00:00\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"2 minutes\" \/>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Beyond Assistants: How AI Could Enable Surgical Robots to Think and Act Autonomously - Department of Electrical and Computer Engineering","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/engineering.jhu.edu\/ece\/news\/beyond-assistants-how-ai-could-enable-surgical-robots-to-think-and-act-autonomously\/","og_locale":"en_US","og_type":"article","og_title":"Beyond Assistants: How AI Could Enable Surgical Robots to Think and Act Autonomously - Department of Electrical and Computer Engineering","og_description":"Surgical robots have long been valuable tools in the operating room, assisting surgeons with precision tasks like suturing, holding instruments, or stabilizing organs. However, these systems still depend heavily on&hellip;","og_url":"https:\/\/engineering.jhu.edu\/ece\/news\/beyond-assistants-how-ai-could-enable-surgical-robots-to-think-and-act-autonomously\/","og_site_name":"Department of Electrical and Computer Engineering","article_modified_time":"2024-12-03T20:43:47+00:00","twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/engineering.jhu.edu\/ece\/news\/beyond-assistants-how-ai-could-enable-surgical-robots-to-think-and-act-autonomously\/","url":"https:\/\/engineering.jhu.edu\/ece\/news\/beyond-assistants-how-ai-could-enable-surgical-robots-to-think-and-act-autonomously\/","name":"Beyond Assistants: How AI Could Enable Surgical Robots to Think and Act Autonomously - Department of Electrical and Computer Engineering","isPartOf":{"@id":"https:\/\/engineering.jhu.edu\/ece\/#website"},"datePublished":"2024-12-03T20:21:26+00:00","dateModified":"2024-12-03T20:43:47+00:00","breadcrumb":{"@id":"https:\/\/engineering.jhu.edu\/ece\/news\/beyond-assistants-how-ai-could-enable-surgical-robots-to-think-and-act-autonomously\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/engineering.jhu.edu\/ece\/news\/beyond-assistants-how-ai-could-enable-surgical-robots-to-think-and-act-autonomously\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/engineering.jhu.edu\/ece\/news\/beyond-assistants-how-ai-could-enable-surgical-robots-to-think-and-act-autonomously\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/engineering.jhu.edu\/ece\/"},{"@type":"ListItem","position":2,"name":"News","item":"https:\/\/engineering.jhu.edu\/ece\/news\/"},{"@type":"ListItem","position":3,"name":"Beyond Assistants: How AI Could Enable Surgical Robots to Think and Act Autonomously"}]},{"@type":"WebSite","@id":"https:\/\/engineering.jhu.edu\/ece\/#website","url":"https:\/\/engineering.jhu.edu\/ece\/","name":"Department of Electrical and Computer Engineering","description":"Department of Electrical and Computer Engineering","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/engineering.jhu.edu\/ece\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"distributor_meta":false,"distributor_terms":false,"distributor_media":false,"distributor_original_site_name":"Department of Electrical and Computer Engineering","distributor_original_site_url":"https:\/\/engineering.jhu.edu\/ece","push-errors":false,"_links":{"self":[{"href":"https:\/\/engineering.jhu.edu\/ece\/wp-json\/wp\/v2\/news\/571571","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/engineering.jhu.edu\/ece\/wp-json\/wp\/v2\/news"}],"about":[{"href":"https:\/\/engineering.jhu.edu\/ece\/wp-json\/wp\/v2\/types\/news"}],"wp:attachment":[{"href":"https:\/\/engineering.jhu.edu\/ece\/wp-json\/wp\/v2\/media?parent=571571"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}