Universität Wien

250063 VO Mathematics of Deep Learning (2024S)

6.00 ECTS (4.00 SWS), SPL 25 - Mathematik
Moodle

An/Abmeldung

Hinweis: Ihr Anmeldezeitpunkt innerhalb der Frist hat keine Auswirkungen auf die Platzvergabe (kein "first come, first served").
Für diese LV an-/abmelden

Details

Sprache: Englisch

Prüfungstermine

Lehrende

Termine (iCal) - nächster Termin ist mit N markiert

  • Montag 04.03. 13:15 - 14:45 Seminarraum 16 Oskar-Morgenstern-Platz 1 3.Stock
  • Donnerstag 07.03. 13:15 - 14:45 Seminarraum 8 Oskar-Morgenstern-Platz 1 2.Stock
  • Montag 11.03. 13:15 - 14:45 Seminarraum 16 Oskar-Morgenstern-Platz 1 3.Stock
  • Donnerstag 14.03. 13:15 - 14:45 Seminarraum 8 Oskar-Morgenstern-Platz 1 2.Stock
  • Montag 18.03. 13:15 - 14:45 Seminarraum 16 Oskar-Morgenstern-Platz 1 3.Stock
  • Donnerstag 21.03. 13:15 - 14:45 Seminarraum 8 Oskar-Morgenstern-Platz 1 2.Stock
  • Montag 08.04. 13:15 - 14:45 Seminarraum 16 Oskar-Morgenstern-Platz 1 3.Stock
  • Donnerstag 11.04. 13:15 - 14:45 Seminarraum 8 Oskar-Morgenstern-Platz 1 2.Stock
  • Montag 15.04. 13:15 - 14:45 Seminarraum 16 Oskar-Morgenstern-Platz 1 3.Stock
  • Donnerstag 18.04. 13:15 - 14:45 Seminarraum 8 Oskar-Morgenstern-Platz 1 2.Stock
  • Montag 22.04. 13:15 - 14:45 Seminarraum 16 Oskar-Morgenstern-Platz 1 3.Stock
  • Donnerstag 25.04. 13:15 - 14:45 Seminarraum 8 Oskar-Morgenstern-Platz 1 2.Stock
  • Montag 29.04. 13:15 - 14:45 Seminarraum 16 Oskar-Morgenstern-Platz 1 3.Stock
  • Donnerstag 02.05. 13:15 - 14:45 Seminarraum 8 Oskar-Morgenstern-Platz 1 2.Stock
  • Montag 06.05. 13:15 - 14:45 Seminarraum 16 Oskar-Morgenstern-Platz 1 3.Stock
  • Montag 13.05. 13:15 - 14:45 Seminarraum 16 Oskar-Morgenstern-Platz 1 3.Stock
  • Donnerstag 16.05. 13:15 - 14:45 Seminarraum 8 Oskar-Morgenstern-Platz 1 2.Stock
  • Donnerstag 23.05. 13:15 - 14:45 Seminarraum 8 Oskar-Morgenstern-Platz 1 2.Stock
  • Montag 27.05. 13:15 - 14:45 Seminarraum 16 Oskar-Morgenstern-Platz 1 3.Stock
  • Montag 03.06. 13:15 - 14:45 Seminarraum 16 Oskar-Morgenstern-Platz 1 3.Stock
  • Donnerstag 06.06. 13:15 - 14:45 Seminarraum 8 Oskar-Morgenstern-Platz 1 2.Stock
  • Montag 10.06. 13:15 - 14:45 Seminarraum 16 Oskar-Morgenstern-Platz 1 3.Stock
  • Donnerstag 13.06. 13:15 - 14:45 Seminarraum 8 Oskar-Morgenstern-Platz 1 2.Stock
  • Montag 17.06. 13:15 - 14:45 Seminarraum 16 Oskar-Morgenstern-Platz 1 3.Stock
  • Donnerstag 20.06. 13:15 - 14:45 Seminarraum 8 Oskar-Morgenstern-Platz 1 2.Stock
  • Montag 24.06. 13:15 - 14:45 Seminarraum 16 Oskar-Morgenstern-Platz 1 3.Stock
  • Donnerstag 27.06. 13:15 - 14:45 Seminarraum 8 Oskar-Morgenstern-Platz 1 2.Stock

Information

Ziele, Inhalte und Methode der Lehrveranstaltung

The lecture attempts to teach multiple facets of deep learning theory. Concretely, we will study the following:

1. What is Deep Learning: A gentle introduction into the language, notation, and main concepts of deep learning on a high level.
2. Feed-forward neural networks: The main building block of deep learning is that of a neural network. We will introduce this in a formal way.
3. Universal Approximation: We will study multiple results that study the (absence of) limitations of deep neural networks to represent general functions.
4. Connection to Splines: There are close relationships between neural networks and classical approximation methods. We will discuss these.
5. ReLU neural networks: There is a special subclass of neural networks that is the most frequently used. We study this special case in more detail.
6. Affine pieces of ReLU neural networks: A standard tool to understand what deep neural networks can and cannot do is to count the number of affine regions that they can generate. We will find upper and lower bounds for this.
7. Deep ReLU neural networks: We study the effect of depth, specifically for deep ReLU neural networks and find reasons for the fact that in practice deeper architectures reign supreme.
8. Curse of Dimensionality: We study the phenomenon of high dimensional approximation, which neural networks seemingly do well, despite the fact that this is typically very hard.
9. Interpolation: Deep neural networks can interpolate data under certain assumptions. We will formalize those.
10. Training of deep neural networks: We will understand how to train neural networks, and under which conditions this training can work
11. Loss landscape analysis: The optimization problem can be understood by studying the topography of the so-called loss landscape.
12. Neural network spaces: The set of all neural networks with a fixed architecture will be studied and conclusions for optimization will be drawn.
13. Generalization: We will describe general statistical learning theory in the context of deep neural networks. This will show under which conditions one can generalize from training sets to test sets.
14. Generalization in the overparameterized regime: The most common regime, i.e., overparameterized is excluded from the results before. We will describe why in practice things still work
15. Adversarial examples and Robustness: Finally, we study under which conditions deep neural networks are robust to changes in their inputs.

Art der Leistungskontrolle und erlaubte Hilfsmittel

Depending on the number of students that take this class, we will have either a written exam or an oral exam at the end of the lecture.

Mindestanforderungen und Beurteilungsmaßstab

To pass this class a student needs to demonstrate that they have understood the material of the course on a basic level. For the best grade, a thorough understanding of each result is necessary.

Prüfungsstoff

Everything that was said in the lecture.

Literatur

There will be course notes that will be published throughout the course.

Zuordnung im Vorlesungsverzeichnis

MAMV; MANV; MSTV;

Letzte Änderung: Fr 18.10.2024 13:46