250042 VU Mathematics of Machine Learning (2021S)

We will introduce the basic concepts of the mathematics behind machine learning. This lecture deals with classical machine learning as compared to deep learning, which is the topic of another lecture.

Topics include:

1. PAC Theory: PAC Learning model, finite hypothesis sets, consistent and inconsistent problems, deterministic and agnostic learning,
2. Rademacher complexity and VC dimension: generalization bounds for Rademacher, Growth function, Connection to Rademacher compl., VC dimension, VC dimension based upper bounds,
lower bounds on generalization.
3. Model Selection: Bias Variance trade-off, Structural Risk minimisation, Cross validation, regularisation
4. Support Vector Machines: generalisation bounds, margin theory/margin based generalization bounds
5. Kernel Methods: Reproducing Kernel Hilbert spaces, Representer Theorem, kernel SVM, generalisation bounds for kernel based methods
6. Clustering: k-means, Lloyds algorithm, Ncut, Cheeger cut, spectral clustering.
7. Dimensionality Reduction: PCA, diffusion maps, Johnson - Lindenstrauss)
8. Neural Networks (Mostly shallow)