{"id":570636,"date":"2024-07-25T15:44:47","date_gmt":"2024-07-25T19:44:47","guid":{"rendered":"https:\/\/engineering.jhu.edu\/ece\/?post_type=news&#038;p=570636"},"modified":"2025-06-02T10:57:59","modified_gmt":"2025-06-02T14:57:59","slug":"jedi-masters-the-art-of-ai-imagery","status":"publish","type":"news","link":"https:\/\/engineering.jhu.edu\/ece\/news\/jedi-masters-the-art-of-ai-imagery\/","title":{"rendered":"JeDi Masters the Art of AI Imagery"},"content":{"rendered":"<p>In a galaxy not so far, far away\u2014Johns Hopkins\u2019 Whiting School of Engineering\u2014researchers have created JeDi, a new approach to personalized text-to-image generation that uses AI to allow users to produce customized images simply by typing what they want to see. What sets JeDi (Joint-Image Diffusion) apart from other such tools is its ability to remember details across the creation of multiple scenes, producing consistent, quality images quickly and efficiently.<\/p>\n<p>\u201cIf you ask JeDI to create a picture of a Golden Retriever sleeping in front of the fireplace and another of a Golden Retriever catching a frisbee in the park, JeDi makes sure it is the same dog\u2014same size, fur, etc.\u2014in both images,\u201d said co-creator Yu Zeng, a graduate student in the <a href=\"https:\/\/engineering.jhu.edu\/ece\/\">Department of Electrical and Computer Engineering<\/a>.<\/p>\n<p>The team presented\u00a0<a href=\"https:\/\/openaccess.thecvf.com\/content\/CVPR2024\/html\/Zeng_JeDi_Joint-Image_Diffusion_Models_for_Finetuning-Free_Personalized_Text-to-Image_Generation_CVPR_2024_paper.html\">\u201c<span>JeDi: Joint-Image Diffusion Models for Finetuning-Free Personalized Text-to-Image Generation<\/span>\u201c<\/a> at the 2024 IEEE\/CVF Conference on Computer Vision and Pattern Recognition last month.<\/p>\n<p>A key factor behind JeDi\u2019s improved performance is the way it was trained. \u201cWhereas traditional text-to-image systems learn from single images, JeDi is trained on a special dataset called S3 (Synthetic Same-Subject), which contains sets of images of the same subjects in different poses and contexts\u201d, says co-creator <a href=\"https:\/\/engineering.jhu.edu\/faculty\/vishal-patel\/\">Vishal Patel<\/a>, associate professor of electrical and computer engineering and instructor in the <a href=\"https:\/\/ep.jhu.edu\/programs\/electrical-engineering-computer-engineering\/\">Engineering for Professionals\u2019 ECE program<\/a>.<\/p>\n<p>\u201cAs a result, JeDi learns to understand the relationships between multiple descriptions and pictures of the same subject. For example, it learns from various images of red apples, along with different descriptions, such as \u2018a shiny red apple\u2019 and \u2018a Red Delicious apple.\u2019 This helps JeDi generate sets of related images, rather than just single images,\u201d Patel said.<\/p>\n<p>Experimental results show JeDi achieves state-of-the-art quality, surpassing existing methods\u2014whether they use specialized training or not\u2014even when given few reference images to learn from.<\/p>\n<p>\u201cFuture research aims to combine JeDi with <a href=\"https:\/\/zengyu.me\/scenec\">SceneComposer<\/a> to simplify image creation,\u201d said Zeng. \u201cUsers will be able to create images by \u2018painting\u2019 with brushes defined by text prompts or example images. Additionally, combining JeDi with fine-tuning could improve personalized image creation, especially with many example images, making the process more efficient and effective.\u201d<\/p>\n<p>Other contributors to the study included Haochen Wang from the Toyota Technological Institute, with Xun Huang, Ting-Chun Wang, Ming-Yu Liu, and Yogesh Balaji from NVIDIA Research.<\/p>\n","protected":false},"template":"","class_list":["post-570636","news","type-news","status-publish","hentry","news_categories-department-news","news_categories-research"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.7 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>JeDi Masters the Art of AI Imagery - Department of Electrical and Computer Engineering<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/engineering.jhu.edu\/ece\/news\/jedi-masters-the-art-of-ai-imagery\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"JeDi Masters the Art of AI Imagery - Department of Electrical and Computer Engineering\" \/>\n<meta property=\"og:description\" content=\"In a galaxy not so far, far away\u2014Johns Hopkins\u2019 Whiting School of Engineering\u2014researchers have created JeDi, a new approach to personalized text-to-image generation that uses AI to allow users to&hellip;\" \/>\n<meta property=\"og:url\" content=\"https:\/\/engineering.jhu.edu\/ece\/news\/jedi-masters-the-art-of-ai-imagery\/\" \/>\n<meta property=\"og:site_name\" content=\"Department of Electrical and Computer Engineering\" \/>\n<meta property=\"article:modified_time\" content=\"2025-06-02T14:57:59+00:00\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"2 minutes\" \/>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"JeDi Masters the Art of AI Imagery - Department of Electrical and Computer Engineering","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/engineering.jhu.edu\/ece\/news\/jedi-masters-the-art-of-ai-imagery\/","og_locale":"en_US","og_type":"article","og_title":"JeDi Masters the Art of AI Imagery - Department of Electrical and Computer Engineering","og_description":"In a galaxy not so far, far away\u2014Johns Hopkins\u2019 Whiting School of Engineering\u2014researchers have created JeDi, a new approach to personalized text-to-image generation that uses AI to allow users to&hellip;","og_url":"https:\/\/engineering.jhu.edu\/ece\/news\/jedi-masters-the-art-of-ai-imagery\/","og_site_name":"Department of Electrical and Computer Engineering","article_modified_time":"2025-06-02T14:57:59+00:00","twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/engineering.jhu.edu\/ece\/news\/jedi-masters-the-art-of-ai-imagery\/","url":"https:\/\/engineering.jhu.edu\/ece\/news\/jedi-masters-the-art-of-ai-imagery\/","name":"JeDi Masters the Art of AI Imagery - Department of Electrical and Computer Engineering","isPartOf":{"@id":"https:\/\/engineering.jhu.edu\/ece\/#website"},"datePublished":"2024-07-25T19:44:47+00:00","dateModified":"2025-06-02T14:57:59+00:00","breadcrumb":{"@id":"https:\/\/engineering.jhu.edu\/ece\/news\/jedi-masters-the-art-of-ai-imagery\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/engineering.jhu.edu\/ece\/news\/jedi-masters-the-art-of-ai-imagery\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/engineering.jhu.edu\/ece\/news\/jedi-masters-the-art-of-ai-imagery\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/engineering.jhu.edu\/ece\/"},{"@type":"ListItem","position":2,"name":"News","item":"https:\/\/engineering.jhu.edu\/ece\/news\/"},{"@type":"ListItem","position":3,"name":"JeDi Masters the Art of AI Imagery"}]},{"@type":"WebSite","@id":"https:\/\/engineering.jhu.edu\/ece\/#website","url":"https:\/\/engineering.jhu.edu\/ece\/","name":"Department of Electrical and Computer Engineering","description":"Department of Electrical and Computer Engineering","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/engineering.jhu.edu\/ece\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"distributor_meta":false,"distributor_terms":false,"distributor_media":false,"distributor_original_site_name":"Department of Electrical and Computer Engineering","distributor_original_site_url":"https:\/\/engineering.jhu.edu\/ece","push-errors":false,"_links":{"self":[{"href":"https:\/\/engineering.jhu.edu\/ece\/wp-json\/wp\/v2\/news\/570636","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/engineering.jhu.edu\/ece\/wp-json\/wp\/v2\/news"}],"about":[{"href":"https:\/\/engineering.jhu.edu\/ece\/wp-json\/wp\/v2\/types\/news"}],"wp:attachment":[{"href":"https:\/\/engineering.jhu.edu\/ece\/wp-json\/wp\/v2\/media?parent=570636"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}