Introduction to Machine Learning (Autumn 2024)

Abstract

The course is an introductory course to machine learning its mathematical foundation.

 We recommend that you do not take this cour if you have already taken the course Machine Learning on the Bachelor programme in Data Science at ITU or a similar.

Description

Machine learning proficiency is increasingly vital in software development processes, both in creating new ML methods and applying them contextually. This course, intended for students with a bachelor's level in software development or similar, delves into the essentials of machine learning models: their development, training, evaluation, and implementation.

Key topics covered include:

• Regression
• Classification
• Clustering
• Dimensionality Reduction

Throughout the course the concepts of 'model', 'learning', and 'evaluation' in machine learning are essential. The curriculum begins with basic machine learning principles, focusing on linear models and linear optimization, complemented by an introduction to fundamental linear algebra concepts. As the semester advances, we explore more advanced subjects, including affine and polynomial models, and delve into other non-linear models such as neural networks, kernel models alongside non-linear optimization methods.

In this course, students will dive into the mathematical models that form the backbone of machine learning. A solid mathematical foundation is indispensable and cannot be bypassed. During the course the students need to focus on:

• Understanding and applying mathematical concepts essential to machine learning models.
• The techniques to optimally estimate model parameters.
• Evaluating and bench marking models on specific problems.

Interpretation of this includes:

• Explaining machine learning models mathematically and their practical implementations.
• Students will not have to reimplement existing advanced ML methods like KNN or neural networks; instead, the emphasis will be on utilizing these algorithms effectively and evaluate them.
• Comprehend and articulate the mathematical underpinnings of models, how to optimally determine model parameters, select suitable machine learning models for problems, decipher model outputs (like interpreting principal components), and evaluate and contrast model results.

By the end of this course, students will be able to:

• Explain the concepts of model, model parameters, learning, and evaluation.
• Distinguish between linear, affine, and non-linear models.
• Understand and explain common loss functions, including linear and non-linear least squares, and regularization.
• Differentiate linear and non-linear optimization principles such as projections, matrix inverse, pseudoinverse, and gradient descent.
• Apply selected linear algebra concepts, including matrix operations, transformations, solving linear equations, projection, basis, and subspaces.
• Perform data wrangling, encompassing data transformations, normalization, standardization, filtering, and convolutions.
• Describe elementary multivariate statistics (mean, variance, covariance) and their application in machine learning.
• Calculate and explain multivariate differentiation and gradients, including their use in model optimization and the chain rule.
• Explain machine learning techniques like regression, classification, dimensionality reduction, and clustering, covering neural networks, support vector machines, principal component analysis, autoencoders, k-means, and mean-shift algorithms.
• Explain concepts like overfitting, underfitting, bias, variance, and bias-variance decomposition, including their link to regularization.
• Implement evaluation metrics for machine learning models, including L-norms, least squares, confusion matrix, precision, and recall.
• Be able program simple machine learning tasks and evaluations in Python.
  
• Utilize Python, Numpy, and PyTorch for machine learning models, including basic image processing and color image handling.

During the weekly exercises and the 2-4 mandatory assignments, students will gain hands-on experience applying the discussed techniques. The focus will be on using, rather than reimplementing, advanced methods for analysis and interpretation. Python and tools like PyTorch and Numpy will be used, along with various datasets, to bring the theories to life through image analysis and machine learning.

Note: Both exercises and mandatory assignments are a part of the exam content.

In this course, in-class exercises are embedded into the lectures, allowing students to apply newly presented concepts. These exercises primarily focus on key mathematical principles and practical programming tasks. To maximize benefit from these sessions, students are expected to thoroughly review the provided material beforehand.

Formal prerequisites

The course assumes the student has the ability to program in some imperative programming language (Java, Python, C/C++/C#.  You should be  a rather proficient programmer as expected on 1. semester on computer science. The course uses concepts from mathematics and in particular linear algebra. Fulfilling the general admittance requirements for CS and SWU ensures the mathematical prerequisites. That is the course presupposes knowledge of Level A high school mathematics and discrete mathematics and includes:

• Vectors in 2D and 3D, covering dot/inner product, cross-product, angles, and distances.
• Differentiation of single-variable functions, with rules for addition, products, and composition.
• Understanding of sums and sequences from discrete mathematics.
• Basic descriptive statistics for single variables (mean, variance, median, mode, histograms)."

We advice that students refresh their memory on the "distance formula", basic trigonometry, vectors, rules of  differentiation (e.g. chain rule), formulas for straight lines and polynomial, prior to the course.
The course suggest students to use the Grasple ("Book of Grasple") for practicing the prerequisite material prior to semester start as provided on learnIT

Intended learning outcomes

After the course, the student should be able to:

• Define, describe and relate concepts, mechanisms and mathematical models underpinning machine learning (ML) methods.
• Analyze and explain key aspects of constructing medium-sized machine learning applications.
• Explain, design and implement classification, regression, clustering and dimensionality reduction methods and apply them to specific problems
• Evaluate, select and adapt appropriate machine learning techniques by applying the theoretical concepts and practical techniques from the course and evaluate the performance.
• Clearly explain and employ basic linear algebra to machine learning.
• Apply the theory and implement rudimentary research papers within machine learning as expected on a master level
• Be able to describe linear and non-linear models of machine learning and how the model parameters are optimally found.

Ordinary exam

Exam type:
B: Oral exam, External (7-point scale)
Exam variation:
B22: Oral exam with no time for preparation.