Advanced Machine Learning (Autumn 2024)

Abstract

This course introduces Machine Learning methods and how they are used in active research and industry applications. Methods will be described and hands-on skills taught in a way to enable you to select the suitable tool for given use-case, and adapt accordingly.

Description

We build upon knowledge from the course "Linear Algebra and Probability", and "Introduction to Machine Learning" or equivalent courses. The course enables students to understand and implement advanced machine learning algorithms, and modify the learnt methods to analyze the outcome of various applications on different datasets. This includes the ability to select appropriate tools and reflect on the results.  

The course is run in two tracks, where we focus on Advanced Machine Learning for Computer Vision (CV) and Advanced Machine Learning for Natural Language Processing (NLP).
In particular in AML for CV we cover traditional and deep learning methods tailored to the visual domain, including but not limited to the following
•    Methods for labelled, unlabelled, and partly labelled data,
•    Traditional image processing vs. neural network methods,
•    Deterministic and probabilistic methods,
•    Discriminative and generative models, including variational, adversarial, and energy-based methods,
•    3D reconstruction from 2D.

While in AML for NLP we cover:
•    supervised learning, including deep neural networks, discriminative learning,
•    regularization, optimization, augmentation, transfer learning,
•    sequence learning including recurrent neural networks, and attention,
•    NLP applications, including language modelling, translation, summarization, and bio-inspired inductive biases,
•    geometric deep learning, including graph neural networks,
•    hybrid learning, adaptive learning, meta-learning.

In the accompanying exercises and mandatory activities, we practise:
●    reproduce key approaches in ML frameworks such as PyTorch and Tensorflow,
●    monitor and analyse training and representation of ML approaches,
●    collaboratively develop & visualise ML approaches,
●    develop a complete ML solution on a task of choice in a small group.

Formal prerequisites

Linear Algebra and Probability
Introduction to Machine Learning

Intended learning outcomes

After the course, the student should be able to:

• Define and describe basic machine learning terms and methods.
• Develop and implement machine learning methods on your own in an appropriate programming language.
• Characterize, relate, and analyze central machine learning concepts and algorithms.
• Combine and modify machine learning methods to analyze practical datasets and reflect on the results.

Learning activities

In both tracks the students will pair in small groups to solve a prepared mini-project of choice (or propose their own project topic) to practice a full machine learning cycle in a collaborative setting.

The track AML for CV will introduce theoretical and practical content during the lectures. Additionally, the accompanying exercise material is designed to deepen the knowledge from the lectures, and help you apply the theory. Note that there will be mandatory activities in the first 10 weeks, and a mini-project in the last 4 weeks which will be part of the exam.
The track AML for NLP will consist of lectures, accompanying exercises, and self-study material to deepen knowledge and follow personal interests with literature recommendations. In the exercises and a mandatory mini-project (not part of the exam), the students will practise concepts, methods, and implementations based on prepared assignments in class.

Mandatory activities

The track AML for CV entails  10 mandatory mini-assignments. In the first 10 weeks, each week there will be one mandatory assignment related to the material of the week. You have to pass 50% of the assignments and actively participate in the mandatory exercise sessions to be admitted to the exam. You will receive feedback from your peers, and TA during the session.
The pedagogical function of the mandatory exercises is to provide you with an activity where you can practice the given ILO’s. E.g. you will directly apply the new theoretical knowledge by implementing machine learning methods, and then analyze and reflect on the results. The students who did not pass the mandatories for good reasons, i.e. failed the first attempt, can request a second attempt from us. The last 4 weeks of the exercise time slots are reserved for the mini-project as part of the exam.

In the track AML for NLP students must develop an ML solution for a task of choice in the form of a mini-project during the last four weeks of the exercises (see learning activities). The project work is necessary to build up practical knowledge in conjunction with the theoretical and conceptual knowledge from the lectures and is not graded but needs to get passed in order to be admitted to the exam. For passing the project, it needs to be submitted (code and reasonable documentation) and presented during the last exercises class, where also feedback is provided by the teacher. In case of a second attempt, the project needs to get presented during an individually arranged meeting with the teacher.
If the mandatory activities are not approved, the student will receive the grade NA (not approved) at the ordinary exam, and will have used an exam attempt.

The student will receive the grade NA (not approved) at the ordinary exam, if the mandatory activities are not approved and the student will use an exam attempt.

Course literature

The following primary sources are available for free online:

• [DL] Deep Learning, Goodfellow, 2017
  http://www.deeplearningbook.org/ 

• [PML] An Introduction to Probabilistic Machine Learning, 2022, Murphy
  https://probml.github.io/pml-book/book1.html 

• [PML2] Probabilistic Machine Learning: Advanced Topics, 2023, Murphy
  https://probml.github.io/pml-book/book2.html 

• [PR] Pattern Recognition and Machine Learning, Christopher Bishop
  https://www.microsoft.com/en-us/research/uploads/prod/2006/01/Bishop-Pattern-Recognition-and-Machine-Learning-2006.pdf 

Additional material will be provided during the course.  

Student Activity Budget

Estimated distribution of learning activities for the typical student

• Preparation for lectures and exercises: 33%
• Lectures: 14%
• Exercises: 19%
• Assignments: 5%
• Project work, supervision included: 10%
• Exam with preparation: 19%

Ordinary exam

Exam type:
D: Submission of written work with following oral, External (7-point scale)
Exam variation:
D2M: Submission for groups or individual with following oral exam supplemented by the submission.

reexam

Exam type:
D: Submission of written work with following oral, External (7-point scale)
Exam variation:
D2M: Submission for groups or individual with following oral exam supplemented by the submission.

Time and date

Ordinary Exam - submission Fri, 10 Jan 2025, 08:00 - 14:00
Ordinary Exam Tue, 21 Jan 2025, 09:00 - 21:00
Ordinary Exam Wed, 22 Jan 2025, 09:00 - 21:00
Ordinary Exam Thu, 23 Jan 2025, 09:00 - 21:00
Ordinary Exam Fri, 24 Jan 2025, 09:00 - 21:00