The proximal augmented Lagrangian method for nonsmooth composite optimization
Abstract:
We develop a primal-dual method for non-smooth convex composite optimization problems in which the objective function is given by the sum of a twice differentiable term and a non-differentiable regularizer. After introducing an auxiliary variable, we utilize the proximal operator of the nonsmooth regularizer to transform the associated augmented Lagrangian into a continuously differentiable function, the proximal augmented Lagrangian. This function is used to develop customized algorithms based on the first and second order primal-dual methods. Our customized method of multipliers algorithm is applicable to a broader class of problems than proximal gradient methods and it has stronger convergence guarantees and a more refined step-size update rules than the alternating direction method of multipliers. When the differentiable component of the objective function is strongly convex with a Lipschitz continuous gradient, we employ the theory of integral quadratic constraints to prove exponential convergence of the primal-descent dual-ascent gradient method. We also use a generalization of the Jacobian to show that the second order updates of the proximal augmented Lagrangian are always a descent direction for the norm of the gradient, and thereby prove that a corresponding continuous-time differential inclusion converges asymptotically to the saddle point. Furthermore, we introduce a merit function to develop a customized algorithm which exhibits superlinear convergence. Finally, we show that in many cases the second order updates can be computed efficiently and provide several examples to demonstrate the effectiveness of our approach. (Joint work with Neil Dhingra and Sei Zhen Khong.)
Biography:
Mihailo Jovanovic (ee.usc.edu/mihailo/) is a professor of Electrical Engineering and the founding director of the Center for Systems and Control at the University of Southern California. He was with the faculty in the Department of Electrical and Computer Engineering at the University of Minnesota, Minneapolis, from 2004 until 2017, and has held visiting positions with Stanford University and the Institute for Mathematics and its Applications. His current research focuses on the design of controller architectures, dynamics and control of fluid flows, and fundamental limitations in the control of large networks of dynamical systems. He serves as the Chair of the APS External Affairs Committee, an Associate Editor of the SIAM Journal on Control and Optimization, and had served as a Program Vice-Chair of the 55th IEEE Conference on Decision and Control and an Associate Editor of the IEEE Control Systems Society Conference Editorial Board. Prof. Jovanovic received a CAREER Award from the National Science Foundation in 2007, the George S. Axelby Outstanding Paper Award from the IEEE Control Systems Society in 2013, and the Distinguished Alumni Award from UC Santa Barbara in 2014.