Advanced Machine Learning for Data Science (Spring 2023)
Official course description:
Course info
Programme
Staff
Course semester
Exam
Abstract
In this course, we teach both advanced machine learning (ML) approaches and hands-on skills for applying these approaches to data science problems.
Description
We teach the current state of the art of Machine Learning (ML) and applying ML to interesting data science problems. The module builds on top of the BSc course on Introductions to Machine Learning and focuses on Deep Learning and Probabilistic Machine Learning. On this basis, we cover both, theoretical and computational ML learning concepts and key mechanisms, as well as technical details for programming ML approaches hands-on in leading ML frameworks.
In particular, this course covers:
- A brief recap of fundamental learning types, preprocessing & data representations, and linear models,
- focussed recap of deterministic and probabilistic decision and information theory,
- supervised learning, including deep neural networks, discriminative learning,
- unsupervised learning, including dimensionality reduction, structure learning, and autoencoders,
- sequence learning including recurrent neural networks, and attention,
- regularization, optimization, augmentation, transfer learning,
- representation learning, including generative models,
- geometric deep learning, including graph neural networks,
- hybrid learning, adaptive learning, meta-learning.
Application areas will include data mining, bioinformatics, natural language processing, computer vision, and robot control.
In the accompanying tutorials, we will exercise to:
- reproduce key approaches in ML frameworks such as PyTorch and Tensorflow,
- monitor and analyse training and representation of ML approaches,
- collaboratively develop & visualise ML approaches,
- develop a complete ML solution on a task of choice in a small group.
Formal prerequisites
Having courses on Applied
Statistics, Linear Algebra, and Optimisation completed is required. A good
background in basic Machine Learning and object-oriented and/or interpreted
programming languages such as Python is recommended.
Intended learning outcomes
After the course, the student should be able to:
- define and describe the state of the art approaches in machine learning (ML),
- characterise and analyse key mechanisms in major ML approaches in-depth,
- implement novel ML mechanisms in leading ML frameworks and develop applications for large-scale data science problems.
Learning activities
The course will consist of lectures, accompanying exercises, and self-study material to deepen knowledge and follow personal interests with literature recommendations. In the exercises during the first ten weeks, the students will practise concepts, methods, and implementations based on prepared assignments in class. In the final four weeks, students will pair in small groups to solve a prepared mini-project of choice (or propose their own project topic) to practice a full machine learning cycle in a collaborative setting.
The oral examination will
assess the students’ learning outcomes based on the provided material through
literature and lecture as well as the practised assignments and mini-project.
Mandatory activities
All students must develop an ML solution for a task of choice in the form of a mini-project (see learning activities). The project work is necessary to build up practical knowledge in conjunction with the theoretical and conceptual knowledge from the lectures and is not graded but needs to get passed in order to be admitted to the exam. For passing the project, it needs to be submitted (code and reasonable documentation) and presented during the last exercises class, where also feedback is provided by the teacher. In case of a second attempt, the project needs to get presented during an individually arranged meeting with the teacher.The student will receive the grade NA (not approved) at the ordinary exam, if the mandatory activities are not approved and the student will use an exam attempt.
Course literature
We focus mostly on the following books and will add specific research papers during the lectures:
Primary:
- [M-PML] Murphy, Probabilistic Machine Learning: An Introduction, MIT Press, 2022. Open access link
Secondary:
- [G-DL] Goodfellow, Bengio, Courville. Deep Learning, MIT Press, 2016. Open access link
- [M-PML-A] Murphy, Probabilistic
Machine Learning: Advanced Topics, MIT Press, 2023. Open access link
- [B-PRML] Bishop, Pattern Recognition and Machine Learning, Springer, 2006, Open access link
Student Activity Budget
Estimated distribution of learning activities for the typical student- Preparation for lectures and exercises: 33%
- Lectures: 14%
- Exercises: 24%
- Project work, supervision included: 10%
- Exam with preparation: 19%
Ordinary exam
Exam type:B: Oral exam, External (7-point scale)
Exam variation:
B22: Oral exam with no time for preparation.
30 minutes
reexam
Exam type:B: Oral exam, External (7-point scale)
Exam variation:
B22: Oral exam with no time for preparation.
30 minutes
Time and date
Ordinary Exam Tue, 6 June 2023, 08:00 - 21:00Ordinary Exam Wed, 7 June 2023, 08:00 - 21:00
Ordinary Exam Thu, 8 June 2023, 08:00 - 21:00
Ordinary Exam Fri, 9 June 2023, 08:00 - 21:00
Reexam Thu, 10 Aug 2023, 09:00 - 17:00