Advanced Natural Language Processing and Deep Learning (Autumn 2024)

Abstract

In this course students will learn to apply modern state-of-the-art solutions for natural language processing problems. We go beyond simple classification tasks, and tackle more advanced types of tasks, like generation and structured prediction. 

Description

This course covers advanced natural language processing tasks, models, and setups. The course builds on the Introduction to Natural Language Processing (NLP) and Deep Learning course (BSSEYEP1KU). We will focus on more advanced tasks, including structured prediction (such as finding relations between words), text generation and multi-task learning, all using modern state-of-the-art language models. Furthermore, we will address low resource scenarios, and the students will learn to build NLP models when no training data for the language or language type of interest is available. 

Formal prerequisites

The student should be able to implement algorithms in python. The student must have taken the Introduction to Natural Language Processing (NLP) and Deep Learning course (BSSEYEP1KU) offered in the BSc Data Science program, or an equivalent course covering at least classification and sequence labeling in NLP. 

If the student did not attend an NLP course at ITU or elsewhere, they should request access to the LearnIT page of the Introduction to Natural Language Processing (NLP) and Deep Learning course (BSSEYEP1KU) course and study the materials. Specifically, they should focus on Chapters 4-8 of the textbook (https://web.stanford.edu/~jurafsky/slp3/old_dec21/indexdec21.html).

Intended learning outcomes

After the course, the student should be able to:

• Summarize recent research in NLP
• Present recent research in NLP
• Evaluate and compare a variety of NLP models
• Recommend accurate solutions for a wide range of NLP tasks
• Design and build state-of-the-art solutions for a wide range of NLP tasks and setups
• Formulate a relevant research question, embedded in current literature
• Report the outcomes of a research project, answering a research question

Ordinary exam

Exam type:
D: Submission of written work with following oral, Internal (7-point scale)
Exam variation:
D2G: Submission for groups with following oral exam supplemented by the submission. Shared responsibility for the report.