NLP & Language Model Foundations
Tokenisers, n-gram and neural language models, and text tasks across many languages and scripts.
Track · Semesters 3–4
Language Models & LLM Engineering
From text processing and tokenisation to training a small GPT from scratch, then post-training, alignment and evaluation, and building retrieval-augmented and agentic applications.
Four subjects, in order
Semester 3 · Subject 1 of 4 Semester 3 · Subject 2 of 4 Building and Training LLMs from Scratch
A tokeniser, a pretraining corpus and a GPT-2-style model, trained and costed on one CPU core.
Semester 4 · Subject 3 of 4 Post-training, Alignment and Evaluation of LLMs
Fine-tuning, reward models, PPO, GRPO and DPO, then an honest evaluation of what changed.
Semester 4 · Subject 4 of 4 LLM Applications: Retrieval, RAG and Agents
Search from first principles, retrieval-augmented generation that cites its sources, and agents that use tools safely.
Then, your capstone
Your capstone is a solo project in this track, in five graded milestones.
Capstone Project
A solo project in your track, in five graded milestones from proposal to report, demo and viva.
Before you start the track
The track builds on the whole core: every Semester 1 and 2 subject.
- Linear Algebra & Calculus for ML
- Probability & Statistics for ML
- Foundations of Machine Learning
- Classical AI: Search, Logic & Reasoning
- Data Engineering & Databases for ML
- Neural Networks & Deep Learning
- Optimisation & Training at Scale
- Sequence Models & Transformers
- Responsible AI, Ethics & Governance
Required for NLP & Language Model Foundations
Sequence Models & Transformers
Neural Networks & Deep Learning
Electives
Open to both tracks.
Probabilistic ML & Bayesian Methods
Posteriors exact and approximate: Gaussian processes, variational inference, MCMC and calibrated uncertainty.
Reinforcement Learning Foundations
Bandits, temporal-difference learning, Q-learning, deep Q-networks and policy gradients, on environments you write.
The other track
Prefer Computer Vision & Generative Models? See the Computer vision track