{"id":51217,"date":"2025-01-27T15:27:03","date_gmt":"2025-01-27T20:27:03","guid":{"rendered":"https:\/\/engineering.jhu.edu\/ams\/?post_type=tribe_events&p=51217"},"modified":"2025-06-24T09:38:02","modified_gmt":"2025-06-24T13:38:02","slug":"ams-special-seminar-series-pratik-patil","status":"publish","type":"tribe_events","link":"https:\/\/engineering.jhu.edu\/ams\/event\/ams-special-seminar-series-pratik-patil\/","title":{"rendered":"AMS Special Seminar Series | Pratik Patil"},"content":{"rendered":"

Location: <\/strong>Clark 110<\/p>\n

When:<\/strong>\u00a0February 11th at 3:00 p.m.<\/p>\n

\n

Title:<\/strong> Facets of Regularization in Overparameterized Machine Learning<\/span><\/p>\n

Abstract:<\/strong>\u00a0Modern machine learning often operates in an overparameterized regime in which the number of parameters far exceeds the number of observations. In this regime, models can exhibit surprising generalization behaviors: (1) Models can overfit with zero training error yet still generalize well (benign overfitting); furthermore, in some cases, even adding and tuning explicit regularization can favor no regularization at all (obligatory overfitting). (2) The generalization error can vary non-monotonically with the model or sample size (double\/multiple descent). These behaviors challenge classical notions of overfitting and the role of explicit regularization.<\/p>\n

In this\u00a0talk, I will present theoretical and methodological results related to these behaviors, primarily focusing on the concrete case of ridge regularization. First, I will identify conditions under which the optimal ridge penalty is zero (or even negative) and show that standard techniques such as leave-one-out and generalized cross-validation, when analytically continued, remain uniformly consistent for the generalization error and thus yield the optimal penalty, whether positive, negative, or zero. Second, I will introduce a general framework to mitigate double\/multiple descent in the sample size based on subsampling and ensembling and show its intriguing connection to ridge regularization. As an implication of this connection, I will show that the generalization error of optimally tuned ridge regression is monotonic in the sample size (under mild data assumptions) and mitigates double\/multiple descent. Key to both parts is the role of implicit regularization, either self-induced by the overparameterized data or externally induced by subsampling and ensembling. Finally, I will briefly mention some extensions and variants beyond ridge regularization.<\/p>\n

The\u00a0talk\u00a0will feature joint work with the following collaborators (in alphabetical order): Pierre Bellec, Jin-Hong Du, Takuya Koriyama, Arun Kumar Kuchibhotla, Alessandro Rinaldo, Kai Tan, Ryan Tibshirani, Yuting Wei. The corresponding papers (in\u00a0talk-chronological order) are: optimal ridge landscape (https:\/\/pratikpatil.io\/papers\/ridge-ood.pdf<\/a>), ridge cross-validation (https:\/\/pratikpatil.io\/papers\/functionals-combined.pdf<\/a>), risk monotonization (https:\/\/pratikpatil.io\/papers\/risk-monotonization.pdf<\/a>), ridge equivalences (https:\/\/pratikpatil.io\/papers\/generalized-equivalences.pdf<\/a>), and extensions and variants (https:\/\/pratikpatil.io\/papers\/cgcv.pdf<\/a>,\u00a0https:\/\/pratikpatil.io\/papers\/subagging-asymptotics.pdf<\/a>).<\/p>\n<\/div>\n

<\/div>\n
Bio:\u00a0<\/strong>Pratik Patil is a postdoctoral researcher in Statistics at the University of California, Berkeley. He obtained his PhD in Statistics and Machine Learning from Carnegie Mellon University. His research broadly spans a range of topics at the intersection of statistical machine learning, optimization, and information theory. Much of his recent work focuses on the statistical analysis of machine learning methods in the overparameterized regime, such as bagging, sketching, cross-validation, and model tuning, drawing upon tools from statistical physics and random matrix theory. More details can be found at:\u00a0https:\/\/pratikpatil.io\/<\/a>.<\/div>\n

Zoom link: <\/strong>https:\/\/wse.zoom.us\/j\/92755277282?pwd=iULpLaFnWAcWl6tQYUbeyZaN3zwBzn.1<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

Location: Clark 110 When:\u00a0February 11th at 3:00 p.m. Title: Facets of Regularization in Overparameterized Machine Learning Abstract:\u00a0Modern machine learning often operates in an overparameterized regime in which the number of…<\/p>\n","protected":false},"author":69,"featured_media":0,"template":"","meta":{"_acf_changed":false,"_relevanssi_hide_post":"","_relevanssi_hide_content":"","_relevanssi_pin_for_all":"","_relevanssi_pin_keywords":"","_relevanssi_unpin_keywords":"","_relevanssi_related_keywords":"","_relevanssi_related_include_ids":"","_relevanssi_related_exclude_ids":"","_relevanssi_related_no_append":"","_relevanssi_related_not_related":"","_relevanssi_related_posts":"","_relevanssi_noindex_reason":"","_tribe_events_status":"","_tribe_events_status_reason":"","footnotes":""},"tags":[],"tribe_events_cat":[260],"class_list":["post-51217","tribe_events","type-tribe_events","status-publish","hentry","tribe_events_cat-seminars-and-endowed-lectures","cat_seminars-and-endowed-lectures"],"acf":[],"yoast_head":"\nAMS Special Seminar Series | Pratik Patil | Department of Applied Mathematics and Statistics<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/engineering.jhu.edu\/ams\/event\/ams-special-seminar-series-pratik-patil\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"AMS Special Seminar Series | Pratik Patil | Department of Applied Mathematics and Statistics\" \/>\n<meta property=\"og:description\" content=\"Location: Clark 110 When:\u00a0February 11th at 3:00 p.m. Title: Facets of Regularization in Overparameterized Machine Learning Abstract:\u00a0Modern machine learning often operates in an overparameterized regime in which the number of…\" \/>\n<meta property=\"og:url\" content=\"https:\/\/engineering.jhu.edu\/ams\/event\/ams-special-seminar-series-pratik-patil\/\" \/>\n<meta property=\"og:site_name\" content=\"Department of Applied Mathematics and Statistics\" \/>\n<meta property=\"article:modified_time\" content=\"2025-06-24T13:38:02+00:00\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"2 minutes\" \/>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"AMS Special Seminar Series | Pratik Patil | Department of Applied Mathematics and Statistics","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/engineering.jhu.edu\/ams\/event\/ams-special-seminar-series-pratik-patil\/","og_locale":"en_US","og_type":"article","og_title":"AMS Special Seminar Series | Pratik Patil | Department of Applied Mathematics and Statistics","og_description":"Location: Clark 110 When:\u00a0February 11th at 3:00 p.m. Title: Facets of Regularization in Overparameterized Machine Learning Abstract:\u00a0Modern machine learning often operates in an overparameterized regime in which the number of…","og_url":"https:\/\/engineering.jhu.edu\/ams\/event\/ams-special-seminar-series-pratik-patil\/","og_site_name":"Department of Applied Mathematics and Statistics","article_modified_time":"2025-06-24T13:38:02+00:00","twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/engineering.jhu.edu\/ams\/event\/ams-special-seminar-series-pratik-patil\/","url":"https:\/\/engineering.jhu.edu\/ams\/event\/ams-special-seminar-series-pratik-patil\/","name":"AMS Special Seminar Series | Pratik Patil | Department of Applied Mathematics and Statistics","isPartOf":{"@id":"https:\/\/engineering.jhu.edu\/ams\/#website"},"datePublished":"2025-01-27T20:27:03+00:00","dateModified":"2025-06-24T13:38:02+00:00","breadcrumb":{"@id":"https:\/\/engineering.jhu.edu\/ams\/event\/ams-special-seminar-series-pratik-patil\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/engineering.jhu.edu\/ams\/event\/ams-special-seminar-series-pratik-patil\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/engineering.jhu.edu\/ams\/event\/ams-special-seminar-series-pratik-patil\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/engineering.jhu.edu\/ams\/"},{"@type":"ListItem","position":2,"name":"Events","item":"https:\/\/engineering.jhu.edu\/ams\/events\/"},{"@type":"ListItem","position":3,"name":"AMS Special Seminar Series | Pratik Patil"}]},{"@type":"WebSite","@id":"https:\/\/engineering.jhu.edu\/ams\/#website","url":"https:\/\/engineering.jhu.edu\/ams\/","name":"Hopkins Applied Math & Statistics","description":"Department of Applied Mathematics and Statistics","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/engineering.jhu.edu\/ams\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"distributor_meta":false,"distributor_terms":false,"distributor_media":false,"distributor_original_site_name":"Department of Applied Mathematics and Statistics","distributor_original_site_url":"https:\/\/engineering.jhu.edu\/ams","push-errors":false,"_links":{"self":[{"href":"https:\/\/engineering.jhu.edu\/ams\/wp-json\/wp\/v2\/tribe_events\/51217","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/engineering.jhu.edu\/ams\/wp-json\/wp\/v2\/tribe_events"}],"about":[{"href":"https:\/\/engineering.jhu.edu\/ams\/wp-json\/wp\/v2\/types\/tribe_events"}],"author":[{"embeddable":true,"href":"https:\/\/engineering.jhu.edu\/ams\/wp-json\/wp\/v2\/users\/69"}],"version-history":[{"count":5,"href":"https:\/\/engineering.jhu.edu\/ams\/wp-json\/wp\/v2\/tribe_events\/51217\/revisions"}],"predecessor-version":[{"id":52954,"href":"https:\/\/engineering.jhu.edu\/ams\/wp-json\/wp\/v2\/tribe_events\/51217\/revisions\/52954"}],"wp:attachment":[{"href":"https:\/\/engineering.jhu.edu\/ams\/wp-json\/wp\/v2\/media?parent=51217"}],"wp:term":[{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/engineering.jhu.edu\/ams\/wp-json\/wp\/v2\/tags?post=51217"},{"taxonomy":"tribe_events_cat","embeddable":true,"href":"https:\/\/engineering.jhu.edu\/ams\/wp-json\/wp\/v2\/tribe_events_cat?post=51217"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}