Skip to content

Life-long Machine Learning – Summer 2025

Course Content

The course traverses the fundamentals of lifelong machine learning, spanning methodology of how to effectively learn in the present, remember the past, and anticipate an unknown future. More specific spanned topics include:

  • curriculum learning and assessing task difficulty,
  • domain adaptation and knowledge transfer
  • Remembering past information: the sequential learning problem, algorithms to mitigate catastrophic interference
  • Handling an unknown future: active learning, closed and open world learning, dynamic and modular neural architectures
  • Biological and cognitive underpinnings of lifelong learning
  • Benchmarks and evaluation of lifelong machine learning
  • Software foundations for lifelong machine learning
  • Lifelong machine learning application

Learning Outcome

Machine learning studies the design of models and training algorithms in order to learn how to solve tasks from data. Whereas historically machine learning has concentrated primarily on static predefined training datasets and respective test scenarios, recent advances also take into account the fact that the world is constantly evolving. Upon successful completion of the course, students will have learned to:

  • understand the breath of factors relevant to lifelong machine learning and their biological inspiration
  • design methods to transfer machine knowledge and mitigate interference in continual training
  • go beyond rigid train-validate-test methodology towards assessment of lifecycles
  • deal with unknown future inputs and adapt machines to diverse contexts

Course Materials

Note that slides for parts 1-3 respectively contain accumulated materials spanning multiple lectures and topics

Lecture Introduction – Download slides

Part 1: Learning in the Present – Download slides

Part 2.1: Remembering the Past: Optimization and Memory – Download slides

Part 2.2: Remembering the Past: Architectures – Download slides

Part 3: Towards the Future: Active & Open World Learning – Download slides

Tutorial Materials:

All tutorial materials are available on https://github.com/OWL-ML/LLML25-tutorial_notebooks

Tutorial 1: Curriculum Learning – due for the tutorial on Wednesday April 23, 2025

Tutorial 2: Transfer Learning – due for the tutorial on Wednesday May 7, 2025

Tutorial 3: EWC and Knowledge Distillation – due for the tutorial on Wednesday May 21, 2025

Tutorial 4: Replay – due for the tutorial on Wednesday June 4th, 2025

Tutorial 5: Progressive Networks – due for the tutorial on Wednesday June 18th, 2025

Tutorial 6: Active Learning – due for the tutorial on Wednesday July 2nd, 2025