Advanced Machine Learning for Data Science (Spring 2023)

Abstract

In this course, we teach both advanced machine learning (ML) approaches and hands-on skills for applying these approaches to data science problems.

Description

We teach the current state of the art of Machine Learning (ML) and applying ML to interesting data science problems. The module builds on top of the BSc course on Introductions to Machine Learning and focuses on Deep Learning and Probabilistic Machine Learning. On this basis, we cover both, theoretical and computational ML learning concepts and key mechanisms, as well as technical details for programming ML approaches hands-on in leading ML frameworks.

In particular, this course covers:

• A brief recap of fundamental learning types, preprocessing & data representations, and linear models,
• focussed recap of deterministic and probabilistic decision and information theory,
• supervised learning, including deep neural networks, discriminative learning,
• unsupervised learning, including dimensionality reduction, structure learning, and autoencoders,
• sequence learning including recurrent neural networks, and attention,
• regularization, optimization, augmentation, transfer learning,
• representation learning, including generative models,
• geometric deep learning, including graph neural networks,
• hybrid learning, adaptive learning, meta-learning.

Application areas will include data mining, bioinformatics, natural language processing, computer vision, and robot control.

In the accompanying tutorials, we will exercise to:

• reproduce key approaches in ML frameworks such as PyTorch and Tensorflow,
• monitor and analyse training and representation of ML approaches,
• collaboratively develop & visualise ML approaches,
• develop a complete ML solution on a task of choice in a small group.

Formal prerequisites

Having courses on Applied Statistics, Linear Algebra, and Optimisation completed is required. A good background in basic Machine Learning and object-oriented and/or interpreted programming languages such as Python is recommended.

Intended learning outcomes

After the course, the student should be able to:

• define and describe the state of the art approaches in machine learning (ML),
• characterise and analyse key mechanisms in major ML approaches in-depth,
• implement novel ML mechanisms in leading ML frameworks and develop applications for large-scale data science problems.

Learning activities

The course will consist of lectures, accompanying exercises, and self-study material to deepen knowledge and follow personal interests with literature recommendations. In the exercises during the first ten weeks, the students will practise concepts, methods, and implementations based on prepared assignments in class. In the final four weeks, students will pair in small groups to solve a prepared mini-project of choice (or propose their own project topic) to practice a full machine learning cycle in a collaborative setting.

The oral examination will assess the students’ learning outcomes based on the provided material through literature and lecture as well as the practised assignments and mini-project.

Mandatory activities

All students must develop an ML solution for a task of choice in the form of a mini-project (see learning activities). The project work is necessary to build up practical knowledge in conjunction with the theoretical and conceptual knowledge from the lectures and is not graded but needs to get passed in order to be admitted to the exam. For passing the project, it needs to be submitted (code and reasonable documentation) and presented during the last exercises class, where also feedback is provided by the teacher. In case of a second attempt, the project needs to get presented during an individually arranged meeting with the teacher. 

The student will receive the grade NA (not approved) at the ordinary exam, if the mandatory activities are not approved and the student will use an exam attempt.

Course literature

We focus mostly on the following books and will add specific research papers during the lectures:

Primary:

• [M-PML] Murphy, Probabilistic Machine Learning: An Introduction, MIT Press, 2022. Open access link
  

Secondary:

• [G-DL] Goodfellow, Bengio, Courville. Deep Learning, MIT Press, 2016. Open access link
  
• [M-PML-A] Murphy, Probabilistic Machine Learning: Advanced Topics, MIT Press, 2023. Open access link
  
• [B-PRML] Bishop, Pattern Recognition and Machine Learning, Springer, 2006, Open access link

Student Activity Budget

Estimated distribution of learning activities for the typical student

• Preparation for lectures and exercises: 33%
• Lectures: 14%
• Exercises: 24%
• Project work, supervision included: 10%
• Exam with preparation: 19%

Ordinary exam

Exam type:
B: Oral exam, External (7-point scale)
Exam variation:
B22: Oral exam with no time for preparation.

reexam

Exam type:
B: Oral exam, External (7-point scale)
Exam variation:
B22: Oral exam with no time for preparation.

Time and date