{"id":572256,"date":"2025-03-07T15:00:58","date_gmt":"2025-03-07T20:00:58","guid":{"rendered":"https:\/\/engineering.jhu.edu\/ece\/?post_type=news&p=572256"},"modified":"2025-03-10T18:24:06","modified_gmt":"2025-03-10T22:24:06","slug":"face-to-face-with-ai-evaluating-machines-ability-to-recognize-and-analyze-faces","status":"publish","type":"news","link":"https:\/\/engineering.jhu.edu\/ece\/news\/face-to-face-with-ai-evaluating-machines-ability-to-recognize-and-analyze-faces\/","title":{"rendered":"Face to Face with AI: Evaluating Machines\u2019 Ability to Recognize and Analyze Faces"},"content":{"rendered":"

In a high-stakes poker game, knowing when your opponent is bluffing and being able to read a subtle smile or frown can make the difference between walking away with the pot or going home empty-handed.<\/p>\n

But for AI, decoding these nuanced facial cues, whether for identity verification, bias detection, or facial recognition, remains a major challenge.<\/p>\n

To tackle this, researchers at Johns Hopkins University developed FaceXBench<\/a>, a comprehensive system designed to assess and improve how Multimodal Large Language Models (MLLMs) interpret and analyze faces.<\/p>\n

“FaceXBench evaluates MLLMs across 14 tasks in six key areas,” said co-author Vishal Patel<\/a>, an associate professor in the Whiting School of Engineering\u2019s Department of Electrical and Computer Engineering<\/a>. “It tests bias and fairness by examining whether models perform equally well across different ages, genders, and racial backgrounds, or if they make more mistakes for certain groups. It also assesses face recognition, including identifying faces in high- and low-resolution images, even recognizing celebrities.\u201d<\/p>\n

Additionally, FaceXBench evaluates face authentication by measuring how well models detect deepfakes and prevent spoofing\u2014attempts to trick an AI system into using fake videos, images, or masks to impersonate someone else. The system further tests facial expression recognition, crowd counting, and head position estimation, which makes facial recognition systems more accurate in real-world situations where people are not always facing the camera. Finally, it checks whether models can find and use tools or techniques related to face analysis, like algorithms for tracking faces across different video frames or images.<\/p>\n

The team\u2019s initial findings highlighted gaps in some MLLMs’ ability to interpret and analyze faces effectively, testing 26 open-source models and two proprietary ones, GPT-40 and GeminiPro 1.5.<\/p>\n

\u201cEven the most advanced AI models had trouble spotting deepfakes or counting people in crowds because these tasks require detecting complex patterns. Methods designed to improve performance, like giving extra instructions or guiding the model\u2019s reasoning, didn\u2019t help either,\u201d said co-author Kartik Narayan, a PhD student in computer science. \u201cBias was also an issue, as the models sometimes gave different results based on age, gender, or race.\u201d<\/p>\n

To address these challenges, the team proposes expanding the training data used for MLLMs to improve accuracy, using face-processing APIs, which are sets of tools that allow software systems to communicate for better performance, and developing techniques to reduce bias. This would ensure fairer outcomes in predictions, which involve AI making decisions like identifying individuals, recognizing expressions, or detecting biases.<\/p>\n

\u201cBy identifying these weaknesses and offering a more structured path for improvement, FaceXBench lays the foundation for advanced, ethically responsible, and accurate face-understanding capabilities in MLLMs,\u201d said co-author Vibashan VS, a PhD student in electrical and computer engineering. \u201cAs future researchers refine these models, they move closer to AI systems that can analyze and interpret human faces with the nuance and reliability required for real-world applications.\u201d<\/p>\n

 <\/p>\n

 <\/p>\n","protected":false},"template":"","class_list":["post-572256","news","type-news","status-publish","hentry","news_categories-research"],"acf":[],"yoast_head":"\nFace to Face with AI: Evaluating Machines\u2019 Ability to Recognize and Analyze Faces - Department of Electrical and Computer Engineering<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/engineering.jhu.edu\/ece\/news\/face-to-face-with-ai-evaluating-machines-ability-to-recognize-and-analyze-faces\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Face to Face with AI: Evaluating Machines\u2019 Ability to Recognize and Analyze Faces - Department of Electrical and Computer Engineering\" \/>\n<meta property=\"og:description\" content=\"In a high-stakes poker game, knowing when your opponent is bluffing and being able to read a subtle smile or frown can make the difference between walking away with the…\" \/>\n<meta property=\"og:url\" content=\"https:\/\/engineering.jhu.edu\/ece\/news\/face-to-face-with-ai-evaluating-machines-ability-to-recognize-and-analyze-faces\/\" \/>\n<meta property=\"og:site_name\" content=\"Department of Electrical and Computer Engineering\" \/>\n<meta property=\"article:modified_time\" content=\"2025-03-10T22:24:06+00:00\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"3 minutes\" \/>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Face to Face with AI: Evaluating Machines\u2019 Ability to Recognize and Analyze Faces - Department of Electrical and Computer Engineering","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/engineering.jhu.edu\/ece\/news\/face-to-face-with-ai-evaluating-machines-ability-to-recognize-and-analyze-faces\/","og_locale":"en_US","og_type":"article","og_title":"Face to Face with AI: Evaluating Machines\u2019 Ability to Recognize and Analyze Faces - Department of Electrical and Computer Engineering","og_description":"In a high-stakes poker game, knowing when your opponent is bluffing and being able to read a subtle smile or frown can make the difference between walking away with the…","og_url":"https:\/\/engineering.jhu.edu\/ece\/news\/face-to-face-with-ai-evaluating-machines-ability-to-recognize-and-analyze-faces\/","og_site_name":"Department of Electrical and Computer Engineering","article_modified_time":"2025-03-10T22:24:06+00:00","twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/engineering.jhu.edu\/ece\/news\/face-to-face-with-ai-evaluating-machines-ability-to-recognize-and-analyze-faces\/","url":"https:\/\/engineering.jhu.edu\/ece\/news\/face-to-face-with-ai-evaluating-machines-ability-to-recognize-and-analyze-faces\/","name":"Face to Face with AI: Evaluating Machines\u2019 Ability to Recognize and Analyze Faces - Department of Electrical and Computer Engineering","isPartOf":{"@id":"https:\/\/engineering.jhu.edu\/ece\/#website"},"datePublished":"2025-03-07T20:00:58+00:00","dateModified":"2025-03-10T22:24:06+00:00","breadcrumb":{"@id":"https:\/\/engineering.jhu.edu\/ece\/news\/face-to-face-with-ai-evaluating-machines-ability-to-recognize-and-analyze-faces\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/engineering.jhu.edu\/ece\/news\/face-to-face-with-ai-evaluating-machines-ability-to-recognize-and-analyze-faces\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/engineering.jhu.edu\/ece\/news\/face-to-face-with-ai-evaluating-machines-ability-to-recognize-and-analyze-faces\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/engineering.jhu.edu\/ece\/"},{"@type":"ListItem","position":2,"name":"News","item":"https:\/\/engineering.jhu.edu\/ece\/news\/"},{"@type":"ListItem","position":3,"name":"Face to Face with AI: Evaluating Machines\u2019 Ability to Recognize and Analyze Faces"}]},{"@type":"WebSite","@id":"https:\/\/engineering.jhu.edu\/ece\/#website","url":"https:\/\/engineering.jhu.edu\/ece\/","name":"Department of Electrical and Computer Engineering","description":"Department of Electrical and Computer Engineering","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/engineering.jhu.edu\/ece\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"distributor_meta":false,"distributor_terms":false,"distributor_media":false,"distributor_original_site_name":"Department of Electrical and Computer Engineering","distributor_original_site_url":"https:\/\/engineering.jhu.edu\/ece","push-errors":false,"_links":{"self":[{"href":"https:\/\/engineering.jhu.edu\/ece\/wp-json\/wp\/v2\/news\/572256","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/engineering.jhu.edu\/ece\/wp-json\/wp\/v2\/news"}],"about":[{"href":"https:\/\/engineering.jhu.edu\/ece\/wp-json\/wp\/v2\/types\/news"}],"wp:attachment":[{"href":"https:\/\/engineering.jhu.edu\/ece\/wp-json\/wp\/v2\/media?parent=572256"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}