Introduction to Machine Learning (Autumn 2024)

Abstract

The course is an introductory course to machine learning its mathematical foundation.

 We recommend that you do not take this cour if you have already taken the course Machine Learning on the Bachelor programme in Data Science at ITU or a similar.

Description

The objectives of this course are to provide students with the fundamental knowledge and skills required to design, build, evolve and evaluate (smaller) machine learning programs. The course requires programming skills equivalent to a completed bachelor in software development (SWU) hence substantial programming experience is expected. The course  contains significant amounts of mathematics and the student should expect to spend some time to  get comfortable with the mathematics and how it is used within Machine learning and programming.

Having knowledge within signals and machine learning (e.g. deep learning) are  important skills needed in many modern innovative businesses. The course will introduce specific machine learning techniques and apply them to relevant small-scale problems such as recognition, regression, dimensionality reduction and clustering. Through the course the student should be able to understand the mathematical foundation and the use the techniques in their own applications and within more advanced topics using machine learning.

Contents:
There is a strong synergy between  machine learning, linear algebra and optimization.  Understanding machine learning requires a good understanding of linear algebra and optimization and vice versa, these topics are nicely explained in the context of machine learning.

The course introduces the basic methodology of machine learning in primarily covering linear aspects of

• regression
• classification
• clustering
• dimensionality reduction

and hence relying on a good foundation of linear algebra.  The course will  also touch upon non-linear models and neural networks.

The course  will present  fundamental mathematical models for machine learning as well as techniques to implement them. The in-class exercises will mostly cover fundamental mathematical topics and small hands-on programming exercises. You will in the exercises and mandatory assignments be getting hands-on experience with using the techniques described during the lectures. You will not be reimplementing advanced methods, but will rather use them for analysis and interpretation.  In the exercises you will be using Python and image analysis / machine learning packages such as OpenCV, PyTorch, Numpy. Datasets of various datatypes will be used in the exercises to illustrate the theory.

In particular the course covers:

•  Linear algebra (matrix multiplication, transformations, solving linear set of equations, subspaces and SVD)
•  Basics of images e.g. pixel-based and local processing of images (smoothing, edges, conversion between color spaces) and color image processing.
• Transformations of data and  data wrangling
  
•  Regression, classification, dimensionality reduction and clustering, including neural networks, deep learning, support vector machines and principal component analysis / Auto encoders, k-means and  mean-shift algorithms,   and recommender systems
• Regularization, non-linear optimization (gradient descent), multivariate differentiation including the chain rule.
  
• Python, OpenCV, Numpy,PyTorch

During the course students should particularly focus on  

• the mathematical models underlying modern machine learning and computer vision techniques. The course will contain significant amounts of mathematics which is essential for understanding the machine learning models.  IT IS ABSOLUTELY not something that can be avoided.
• The techniques for estimating the parameters of the model given the definition of optimality.
• evaluating  and comparing models on a given problem using established
  

Interpretation
This means that you should be able to use and explain the mathematics that is taught in the course. Similarly you should be able to explain the models of machine learning in terms of mathematics and indicate how they can be implemented, but (and this is important) you will NOT be asked to reimplement specific / advanced ML methods (such as KNN, neural networks etc). There already exist very good implementations and no need to make your own implementation. It is MUCH more important that you can

• understand and describe the underlying (mathematical) models and how model parameters are optimally found.
  

• select an appropriate Machine learning model for a given problem.

• interpret the output of the model (eg. what do the principal components represent)

• compare and evaluate the results from the model

Formal prerequisites

The course assumes the student has the ability to program in some imperative programming language (Java, Python, C/C++/C#.  You should be  a rather proficient programmer as expected on 1. semester on computer science. The course uses concepts from mathematics and in particular linear algebra. Fulfilling the general admittance requirements for CS and SWU ensures the mathematical prerequisites. We advice that students refresh their memory on the distance formula, basic trgonometry, vectors, rules of  differentiation (e.g. chain rule), formulas for straight lines and polynomia, prior to the course.

Intended learning outcomes

After the course, the student should be able to:

• Define, describe and relate concepts, mechanisms and mathematical models underpinning machine learning (ML) methods.
• Analyze and explain key aspects of constructing medium-sized machine learning applications.
• Explain, design and implement classification, regression, clustering and dimensionality reduction methods and apply them to specific problems
• Evaluate, select and adapt appropriate machine learning techniques by applying the theoretical concepts and practical techniques from the course and evaluate the performance.
• Clearly explain and employ basic linear algebra to machine learning.
• Apply the theory and implement rudimentary research papers within machine learning as expected on a master level
• Be able to describe linear and non-linear models of machine learning and how the model parameters are optimally found.

Ordinary exam