{"id":49001,"date":"2024-08-26T10:23:59","date_gmt":"2024-08-26T14:23:59","guid":{"rendered":"https:\/\/engineering.jhu.edu\/ams\/?post_type=tribe_events&p=49001"},"modified":"2024-09-09T13:01:56","modified_gmt":"2024-09-09T17:01:56","slug":"ams-weekly-seminar-professor-michael-mahoney","status":"publish","type":"tribe_events","link":"https:\/\/engineering.jhu.edu\/ams\/event\/ams-weekly-seminar-professor-michael-mahoney\/","title":{"rendered":"AMS Weekly Seminar | Michael Mahoney"},"content":{"rendered":"

Location:\u00a0<\/strong>Gilman 50<\/p>\n

When:<\/strong> September 12th\u00a0at 1:30 p.m.<\/p>\n

Title:<\/strong> Model Selection And Ensembling When There Are More Parameters Than Data<\/p>\n

Abstract:<\/strong> Despite years of empirical success with deep learning for many large-scale problems, existing theoretical frameworks fail to explain many of the most successful heuristics used by practitioners.\u00a0 The primary weakness most approaches encounter is a reliance on the typical large data regime, which neural networks often do not operate in due to their large size.\u00a0 To overcome this issue, I will describe how for any overparameterized (high-dimensional) model, there exists a dual underparameterized (low-dimensional) model that possesses the same marginal likelihood, establishing a form of Bayesian duality.\u00a0 Applying classical methods to this dual model reveals the Interpolating Information Criterion, a measure of model quality that is consistent with current deep learning heuristics.\u00a0 I will also describe how, in many modern machine learning settings, the benefits of ensembling are less ubiquitous and less obvious than classically.\u00a0 Theoretically, we prove simple new results relating the ensemble improvement rate (a measure of how much ensembling decreases the error rate versus a single model, on a relative scale) to the disagreement-error ratio.\u00a0 Empirically, the predictions made by our theory hold, and we identify practical scenarios where ensembling does and does not result in large performance improvements.\u00a0 Perhaps most notably, we demonstrate a distinct difference in behavior between interpolating models (popular in current practice) and non-interpolating models (such as tree-based methods, where ensembling is popular), demonstrating that ensembling helps considerably more in the latter case than in the former.<\/p>\n

Bio:\u00a0<\/strong>Michael W. Mahoney is at the University of California at Berkeley in the Department of Statistics and at the International Computer Science Institute (ICSI).\u00a0 He is also an Amazon Scholar as well as head of the Machine Learning and Analytics Group at the Lawrence Berkeley National Laboratory.\u00a0 He works on algorithmic and statistical aspects of modern large-scale data analysis.\u00a0 Much of his recent research has focused on large-scale machine learning, including randomized matrix algorithms and randomized numerical linear algebra, scalable stochastic optimization, geometric network analysis tools for structure extraction in large informatics graphs, scalable implicit regularization methods, computational methods for neural network analysis, physics informed machine learning, and applications in genetics, astronomy, medical imaging, social network analysis, and internet data analysis.\u00a0 He received his PhD from Yale University with a dissertation in computational statistical mechanics, and he has worked and taught at Yale University in the mathematics department, at Yahoo Research, and at Stanford University in the mathematics department.\u00a0 Among other things, he was on the national advisory committee of the Statistical and Applied Mathematical Sciences Institute (SAMSI), he was on the National Research Council\u2019s Committee on the Analysis of Massive Data, he co-organized the Simons Institute\u2019s fall 2013 and 2018 programs on the foundations of data science, he ran the Park City Mathematics Institute\u2019s 2016 PCMI Summer Session on The Mathematics of Data, he ran the biennial MMDS Workshops on Algorithms for Modern Massive Data Sets, and he was the Director of the NSF\/TRIPODS-funded FODA (Foundations of Data Analysis) Institute at UC Berkeley.\u00a0 More information is available at https:\/\/www.stat.berkeley.edu\/~mmahoney\/<\/span><\/a>.<\/p>\n

Zoom link:<\/strong> https:\/\/wse.zoom.us\/j\/94601022340<\/p>\n","protected":false},"excerpt":{"rendered":"

Location:\u00a0Gilman 50 When: September 12th\u00a0at 1:30 p.m. Title: Model Selection And Ensembling When There Are More Parameters Than Data Abstract: Despite years of empirical success with deep learning for many…<\/p>\n","protected":false},"author":69,"featured_media":0,"template":"","meta":{"_acf_changed":false,"_relevanssi_hide_post":"","_relevanssi_hide_content":"","_relevanssi_pin_for_all":"","_relevanssi_pin_keywords":"","_relevanssi_unpin_keywords":"","_relevanssi_related_keywords":"","_relevanssi_related_include_ids":"","_relevanssi_related_exclude_ids":"","_relevanssi_related_no_append":"","_relevanssi_related_not_related":"","_relevanssi_related_posts":"","_relevanssi_noindex_reason":"","_tribe_events_status":"","_tribe_events_status_reason":"","footnotes":""},"tags":[],"tribe_events_cat":[260],"class_list":["post-49001","tribe_events","type-tribe_events","status-publish","hentry","tribe_events_cat-seminars-and-endowed-lectures","cat_seminars-and-endowed-lectures"],"acf":[],"yoast_head":"\nAMS Weekly Seminar | Michael Mahoney | Department of Applied Mathematics and Statistics<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/engineering.jhu.edu\/ams\/event\/ams-weekly-seminar-professor-michael-mahoney\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"AMS Weekly Seminar | Michael Mahoney | Department of Applied Mathematics and Statistics\" \/>\n<meta property=\"og:description\" content=\"Location:\u00a0Gilman 50 When: September 12th\u00a0at 1:30 p.m. Title: Model Selection And Ensembling When There Are More Parameters Than Data Abstract: Despite years of empirical success with deep learning for many…\" \/>\n<meta property=\"og:url\" content=\"https:\/\/engineering.jhu.edu\/ams\/event\/ams-weekly-seminar-professor-michael-mahoney\/\" \/>\n<meta property=\"og:site_name\" content=\"Department of Applied Mathematics and Statistics\" \/>\n<meta property=\"article:modified_time\" content=\"2024-09-09T17:01:56+00:00\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"3 minutes\" \/>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"AMS Weekly Seminar | Michael Mahoney | Department of Applied Mathematics and Statistics","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/engineering.jhu.edu\/ams\/event\/ams-weekly-seminar-professor-michael-mahoney\/","og_locale":"en_US","og_type":"article","og_title":"AMS Weekly Seminar | Michael Mahoney | Department of Applied Mathematics and Statistics","og_description":"Location:\u00a0Gilman 50 When: September 12th\u00a0at 1:30 p.m. Title: Model Selection And Ensembling When There Are More Parameters Than Data Abstract: Despite years of empirical success with deep learning for many…","og_url":"https:\/\/engineering.jhu.edu\/ams\/event\/ams-weekly-seminar-professor-michael-mahoney\/","og_site_name":"Department of Applied Mathematics and Statistics","article_modified_time":"2024-09-09T17:01:56+00:00","twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/engineering.jhu.edu\/ams\/event\/ams-weekly-seminar-professor-michael-mahoney\/","url":"https:\/\/engineering.jhu.edu\/ams\/event\/ams-weekly-seminar-professor-michael-mahoney\/","name":"AMS Weekly Seminar | Michael Mahoney | Department of Applied Mathematics and Statistics","isPartOf":{"@id":"https:\/\/engineering.jhu.edu\/ams\/#website"},"datePublished":"2024-08-26T14:23:59+00:00","dateModified":"2024-09-09T17:01:56+00:00","breadcrumb":{"@id":"https:\/\/engineering.jhu.edu\/ams\/event\/ams-weekly-seminar-professor-michael-mahoney\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/engineering.jhu.edu\/ams\/event\/ams-weekly-seminar-professor-michael-mahoney\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/engineering.jhu.edu\/ams\/event\/ams-weekly-seminar-professor-michael-mahoney\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/engineering.jhu.edu\/ams\/"},{"@type":"ListItem","position":2,"name":"Events","item":"https:\/\/engineering.jhu.edu\/ams\/events\/"},{"@type":"ListItem","position":3,"name":"AMS Weekly Seminar | Michael Mahoney"}]},{"@type":"WebSite","@id":"https:\/\/engineering.jhu.edu\/ams\/#website","url":"https:\/\/engineering.jhu.edu\/ams\/","name":"Hopkins Applied Math & Statistics","description":"Department of Applied Mathematics and Statistics","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/engineering.jhu.edu\/ams\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"distributor_meta":false,"distributor_terms":false,"distributor_media":false,"distributor_original_site_name":"Department of Applied Mathematics and Statistics","distributor_original_site_url":"https:\/\/engineering.jhu.edu\/ams","push-errors":false,"_links":{"self":[{"href":"https:\/\/engineering.jhu.edu\/ams\/wp-json\/wp\/v2\/tribe_events\/49001","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/engineering.jhu.edu\/ams\/wp-json\/wp\/v2\/tribe_events"}],"about":[{"href":"https:\/\/engineering.jhu.edu\/ams\/wp-json\/wp\/v2\/types\/tribe_events"}],"author":[{"embeddable":true,"href":"https:\/\/engineering.jhu.edu\/ams\/wp-json\/wp\/v2\/users\/69"}],"version-history":[{"count":6,"href":"https:\/\/engineering.jhu.edu\/ams\/wp-json\/wp\/v2\/tribe_events\/49001\/revisions"}],"predecessor-version":[{"id":49814,"href":"https:\/\/engineering.jhu.edu\/ams\/wp-json\/wp\/v2\/tribe_events\/49001\/revisions\/49814"}],"wp:attachment":[{"href":"https:\/\/engineering.jhu.edu\/ams\/wp-json\/wp\/v2\/media?parent=49001"}],"wp:term":[{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/engineering.jhu.edu\/ams\/wp-json\/wp\/v2\/tags?post=49001"},{"taxonomy":"tribe_events_cat","embeddable":true,"href":"https:\/\/engineering.jhu.edu\/ams\/wp-json\/wp\/v2\/tribe_events_cat?post=49001"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}