- This event has passed.
AMS Weekly Seminar | Shiqian Ma
January 23 @ 8:00 am - 5:00 pm
Location: Gilman 50
When: January, 23rd at 1:30 p.m.
Title: Decentralized Bilevel Optimization
Abstract: Bilevel optimization has gained significant attention in recent years due to its broad applications in machine learning. In this talk we focus on bilevel optimization in decentralized networks. In particular, we propose two algorithms for solving decentralized bilevel optimization. The first algorithm is a hypergradient method in the decentralized network environment, which is the first decentralized algorithm for bilevel optimization. The second algorithm is a novel single-loop algorithm that approximates the hypergradient using only two matrix-vector multiplications per iteration. Our single-loop algorithm does not require any gradient heterogeneity assumptions, which is a significant improvement over existing methods for decentralized bilevel optimization. Our analysis demonstrates that the proposed algorithm achieves the best-known convergence rate for decentralized bilevel optimization algorithms. We present experimental results on hyperparameter optimization and data hyper-cleaning problems to demonstrate the efficiency of our proposed algorithms.
Bio: Shiqian Ma is a professor in the Department of Computational Applied Mathematics and Operations Research and Department of Electrical and Computer Engineering at Rice University. He received his PhD in Industrial Engineering and Operations Research from Columbia University. His main research areas are optimization and machine learning. His research has been supported by ONR and NSF Grants from the DMS, CCF, and ECCS programs. Shiqian received the 2024 INFORMS Computing Society Prize and the 2024 SIAM Review SIGEST Award, among many other awards from both academia and industry. Shiqian is an Associate Editor of SIAM Journal on Optimization, Journal of Machine Learning Research, Journal of Optimization Theory and Applications, Pacific Journal of Optimization, and IISE Transactions, a Senior Area Chair of NeurIPS, an Area Chair of ICML, ICLR and AISTATS, and a Senior Program Committee of AAAI. He is a plenary speaker of the Texas Colloquium on Distributed Learning in 2023 and a semi-plenary speaker of the International Conference on Stochastic Programming in 2023. Shiqian is the elected Secretary/Treasurer of the INFORMS Optimization Society in 2023-2025, and is the General Chair of the INFORMS Optimization Society Conference 2024.
Zoom link: https://wse.zoom.us/j/96299385386?pwd=HvOzDnw7A2SOQkMrbBbr5AXjqurdj0.1