Project description
The art of making coffee is a science in itself. By now, there exist several scientific books on how parameters such as the coarseness of the grind, a machine’s pressure, the extraction time and amounts, or the type of brewing influence the final taste of the made drink. These factors are further augmented by a rich possibility to make different types of coffee drinks (e.g. espresso vs. americano vs. cappuccino) and a wide availability of differently grown and roasted beans. Understanding the combination of these factors is far from straightforward, especially when coupled with subjective taste palettes of individual coffee drinkers.
In the Multi-Objective Continual Coffee Automation (MOCCA) student project, we will make use of various machine learning tools to go from a package of coffee beans to coffee drinks that are catered to individual preferences. This involves several crucial steps that will be pursued in the project under consideration of multiple modalities. On the side of the coffee itself, we will take pictures of the packaging, learn to extract critical information (such as the roast, type, origin and described flavor palette) and translate it to a common representation. For brewing, we will train deep neural networks and fine-tune large existing models, such that the models learn to suggest suitable brewing parameters depending on the coffee. Over time, we will keep investigating new types of beans and will thus need to equip our neural networks with two fundamental capabilities: 1. The ability to understand when a novel type of bean is presented, 2. The ability to continually learn without forgetting past knowledge. Finally, we will naturally get to drink the coffee we make and log our personal and subjective assessment through natural language. For instance, we could say “this coffee is a bit too sour for my taste, but I like the hint of chocolate and I might prefer it with milk”. Our last step will thus be to personalize our deep neural networks to account for the customized taste of each student.
Practically pursued research topics
Our project of coffee automation interfaces several state-of-the-art machine learning research questions that will be deepened in understanding and implemented practically. Importantly, we will encounter the caveats of what makes for a good dataset, how to process and learn from multi-modal data with deep neural networks, how we can augment neural networks with continual learning mechanisms, and how to include preference learning techniques