Foundations of Machine Learning

Three weeks, 15 days, a lecture and exercises every day. The three-week course takes place from 9:00-17:00 at the University IT and Data Center (Hochschulrechenzentrum HRZ). The course structure is 90 minutes of lecture 90 min exercises, followed by 4 hrs of programming under guidance from the tutors.

Prerequisites:
Programming in Python. If you are not yet familiar with python, please consult https://docs.python.org/3/tutorial/ before the first session.

Course contents:

Week I, Basics
      Day 1: Introduction
              - What is machine learning, and what can it do for us?
      Day 2: Optimization
             - The derivative, gradients, optimization via gradient descent.
      Day 3:   Linear Algebra:
            - Matrix multiplication, singular value decomposition, Linear Regression.
      Day 4:  Statistics - Probability Theory I
            - random variable, mean and variance, conditional probability.
      Day 5: Statistics - Probability Theory II
            - common probability distributions, correlation and auto-correlation.

Week 2, Foundations of machine learning
      Day 1: Machine learning basics
              - Overfitting and underfitting, classification, regression, k-nearest neighbours.
      Day 2: Support vector machines
             - Linear separable, non-linear separable, kernel trick.
      Day 3: Decision trees and random forests:
            - Decision trees, random forests, bias and variance problem, bagging.
      Day 4:  Clustering and density estimation
            - K-means clustering, Gaussian mixture models, expectation-maximization.
      Day 5: Principal component analysis (PCA)
            - PCA for dimensionality reduction, PCA for compression and other applications.

Week 3, Deep Learning
      Day 11: Fully connected networks:
              -  The MNIST-data set, artificial neurons, forward and backward pass.
      Day 12: Convolutional neural networks:
              -  The convolution operation and convolutional neural networks.
      Day 13: Optimization for deep neural networks:
              -  gradient descent with momentum, Adam, early stopping, regularization.
      Day 14: Interpretability:
              - visualization of linear classifiers, saliency maps, integrated gradients
      Day 15: Sequence models:
             - Long-Short-Term-Memory, Gated recurrent units, text-based language models.

See you during the course,

Your lecturers, Elena and Moritz.