The Alternating Direction Method of Multipliers: Recent Advances and Applications
After reviewing developments in the Alternating Direction Method of Multipliers (ADMM) over the last 45 years, we present some recent advances in and applications of these methods. Originally, ADMM theory only applied to the minimization of convex functions of two blocks of variables coupled by linear equations. In the last few years ADMM theory has been extended to non-smooth and non-convex functions of multiple blocks of variables. Recently, we have been able to extend these results to multiaffine constrained blocks of variables, enabling ADMM to be applied to problems such as nonnegative matrix factorization, sparse learning, risk parity portfolio selection, polynomial optimization, nonconvex formulations of convex problems, and neural network training. In each case, our multiaffine ADMM approach encounters only sub-problems that have closed-form solutions.