Publications

2025

Scaling Probabilistic Circuits via Data Partitioning

Authors: Jonas Seng, Florian P. Busch, Pooja Prasad, Devendra S. Dhami, Martin Mundt, Kristian Kersting

Short version accepted at: ICLR 2025 Workshop on Modularity for Collaborative, Decentralized, and Continual Deep Learning as “Federated Circuits: A Unified Framework for Scalable and Efficient Federated Learning”

Preprint of long version: arXiv:2503.08141, March 2025

Long Paper

ICLR-Workshop Paper

Code

Abstract: Continual learning (CL) is the sub-field of machine learning concerned with accumulating knowledge in dynamic environments. So far, CL research has mainly focused on incremental classification tasks, where models learn to classify new categories while retaining knowledge of previously learned ones. Here, we argue that maintaining such a focus limits both theoretical development and practical applicability of CL methods. Through a detailed analysis of concrete examples – including multi-target classification, robotics with constrained output spaces, learning in continuous task domains, and higher-level concept memorization – we demonstrate how current CL approaches often fail when applied beyond standard classification. We identify three fundamental challenges: (C1) the nature of continuity in learning problems, (C2) the choice of appropriate spaces and metrics for measuring similarity, and (C3) the role of learning objectives beyond classification. For each challenge, we provide specific recommendations to help move the field forward, including formalizing temporal dynamics through distribution processes, developing principled approaches for continuous task spaces, and incorporating density estimation and generative objectives. In so doing, this position paper aims to broaden the scope of CL research while strengthening its theoretical foundations, making it more applicable to real-world problems.

Continual Learning Should Move Beyond Incremental Classification

Authors: Rupert Mitchell, Antonio Alliegro, Raffaello Camoriano, Dustin Carrión-Ojeda, Antonio Carta, Georgia Chalvatzaki, Nikhil Churamani, Carlo D’Eramo, Samin Hamidi, Robin Hesse, Fabian Hinder, Roshni Kamath, Vincenzo Lomonaco, Subarnaduti Paul, Francesca Pistilli, Tinne Tuytelaars, Gido van de Ven, Kristian Kersting, Simone Schaub-Meyer, Martin Mundt

Preprint: arXiv:2502.11927, February 2025

2025

Scaling Probabilistic Circuits via Data Partitioning

Continual Learning Should Move Beyond Incremental Classification

The Cake that is Intelligence and Who Gets to Bake it: An AI Analogy and its Implications for Participation

2024

Aligning Generalisation Between Humans and Machines

Core Tokensets for Data-efficient Sequential Training of Transformers

Distribution-Aware Replay for Continual MRI Segmentation

Masked Autoencoders are Efficient Continual Federated Learners

Continual Learning: Applications and the Road Forward

Where is the Truth? The Risk of Getting Confounded in a Continual World

Self-Expanding Neural Networks

BOWLL: A Deceptively Simple Open World Lifelong Learner

Deep Classifier Mimicry without Data Access

Adaptive Rational Activations to Boost Deep Reinforcement Learning

Designing a Hybrid Neural System to Learn Real-world Crack Segmentation from Fractal-based Simulation

2023

Continual Causality: A Retrospective of the Inaugural AAAI-23 Bridge Program

Probabilistic Circuits That Know What They Don’t Know

Queer In AI: A Case Study in Community-Led Participatory AI

Benchmarking the Second Generation of Intel SGX for Machine Learning Workloads

FEATHERS: Federated Architecture and Hyperparameter Search

A Wholistic View of Continual Learning with Deep Neural Networks: Forgotten Lessons and the Bridge to Active and Open World Learning

2022

Elevating Perceptual Sample Quality in Probabilistic Circuits through Differentiable Sampling

Return of the normal distribution: Flexible deep continual learning with variational auto-encoders

When Deep Classifiers Agree: Analyzing Correlations between Learning Order and Image Statistics

Towards Coreset Learning in Probabilistic Circuits

Predictive Whittle Networks for Time Series

CLEVA-Compass: A Continual Learning EValuation Assessment Compass to Promote Research Transparency and Comparability

Unified Probabilistic Deep Continual Learning Through Generative Replay and Open Set Recognition