IKT719 Advanced Optimization

University of Agder

Course contents

The course covers the following main topics:

1) Review of optimization algorithms for smooth problems:

-Overview of basic first-order optimization iterative algorithms

-Overview of second-order optimization iterative algorithms

2) Methods for optimization of non-smooth problems:

-Sub-gradient methods for unconstrained and constrained non-smooth convex problems.

-Proximal algorithms

-Localization and cutting-plane methods

3) Distributed and decentralized algorithms:

-Primal and Dual decomposition methods

-Applications of problem decomposition methods

-Alternating direction method of multipliers (ADMM) and Applications

4) Stochastic Optimization methods:

-Stochastic gradient/sub-gradient methods

-Online optimization methods and applications

5) Methods for non-convex problems:

-l1 methods for convex-cardinality non-convex problems and Applications (Lasso, Support Vector Machines, Total Variation, sparsity problems, machine learning)

-Sequential convex programming

The theory and algorithms will be interlaced with several applications in different disciplines: selected applications in areas such as signal processing, data analytics, big data, machine learning, control, circuit design, wireless communication & sensor networks, distributed processing on graphs. These applications will be explored also in the periodic homeworks, which will include both analysis and programming tasks.

Learning outcomes

Upon successful completion of this course, the students should:

- know how to recognize, model and formulate research problems from different fields as convex or non-convex optimization problems.

- understand the underlying theory, concepts and properties related to advanced optimization tools, especially those that are useful for computing solutions efficiently in both convex and non-convex problems.

- be able to design, implement and simulate practical algorithms to solve the various optimization problems.

- be able to analyze the structures of complex optimization problems and the corresponding exact or approximate solutions, as well as the relationships or approximations betwendifferent problems.

Back