269011 VO Numerical Methods III - Optimisation (2021S)

Basic concepts of continuous optimization from theory with simple proofs to (large-scale) numerical methods. Basic knowledge in analysis and (numerical) linear algebra (e.g. Numerical Methods I & II) are helpful but not strictly required.

Topics: line search and trust region algorithms, Newton and (large-scale) Quasi-Newton methods, nonlinear conjugate gradient (NCG) methods, theory of (nonlinear) constrained optimization, linear programming, (sequential) quadratic programming, penalty and augmented Lagrangian methods, interior point methods.
Both theoretical background and practical numerical aspects (e.g. machine learning, python scikit-learn, (nonlinear) dimensionality reduction, etc) will be emphases.

Unfortunately this lecture lacks its accompanying exercise course. Therefore, in addition, exercise examples will be made available, which can be worked out on a voluntary basis.