7CS518 — Natural Language Processing

Overview

This postgraduate module covers the core methods of natural language processing, from statistical language models and n-gram approaches to neural architectures designed for sequential data. Students examine the theoretical foundations and practical implementation of models for text understanding and generation.

Topics

• Statistical language models and n-grams
• Text preprocessing and feature extraction
• Recurrent Neural Networks (RNNs)
• Long Short-Term Memory networks (LSTMs)
• Sequence-to-sequence modelling
• Introduction to transformer-based language models

Companion notebooks

The open course From Words to Meaning: Natural Language Processing in Python covers the topics of this module in 18 hands-on notebooks that run in Google Colab, and goes further into modern language models. It is a general course, not tied to this module, but the table shows where to look each week.

Unit	Topic	Notebooks in From Words to Meaning
1	Introduction to NLP	1
2	Basic text processing in Python	1, 2, 3
4	Language modelling, part-of-speech tagging, named entities	6, 7
5	Text representation	8, 10
6	Formal languages and automata	2, 4, 5
7	Recurrent networks and LSTMs	12
8	Sequence-to-sequence learning	13
9	Convolutional networks for text	14
10	Text from the web	16
12	Summarisation	17

Text classification is covered in notebook 9. Notebooks 11, 15 and 18 go beyond the module: topic modelling, subword tokenisation and decoding, and retrieval-augmented generation.

The machine learning and neural network foundations of units 2, 3 and 11 are covered in From Python to Deep Learning and Fine-Tuning, notebooks 8 to 13.