CMU 10-725 Convex Optimization

Carnegie Mellon University,, Fall 2018 , Prof. Ryan Tibshirani

Updated On 02 Feb, 19

Overview

Nearly every problem in machine learning and computational statistics can be formulated
in terms of the optimization of some function, possibly under some set of constraints. As
we obviously cannot solve every problem in machine learning, this means that we cannot
generically solve every optimization problem (at least not efficiently). Fortunately, many
problems of interest in machine learning can be posed as optimization tasks that have
special propertiessuch as convexity, smoothness, sparsity, separability, etc.permitting
standardized, efficient solution techniques.

This course is designed to give a graduate-level student a thorough grounding in these
properties and their role in optimization, and a broad comprehension of algorithms tailored
to exploit such properties. The focus will be on convex optimization problems (though
we also may touch upon nonconvex optimization problems at some points). We will visit
and revisit important applications in machine learning and statistics.

Upon completing the
course, students should be able to approach an optimization problem (often derived from a
machine learning or statistics context) and:

1. identify key properties such as convexity, smoothness, sparsity, etc., and/or possibly
reformulate the problem so that it possesses such desirable properties;
2. select an algorithm for this optimization problem, with an understanding of the advantages and disadvantages of applying one method over another, given the problem
and properties at hand;
3. implement this algorithm or use existing software to efficiently compute the solution.