Sony AI and Korea University

Among the partnerships we’re setting up to drive our Gastronomy Flagship Project, we’re excited that our work with Korea University (KU) is already pushing our understanding of how AI can meaningfully pair ingredients. A project looking to advance flavor pairing knowledge has resulted in an ingredient mapping tool called FlavorGraph.

Generally speaking, FlavorGraph predicts the pairing fit of two ingredients by combining information on the molecules in a given ingredient with the way people have used that ingredient in the past: as such, FlavorGraph sits on the data of 1,561 flavor molecules and of a million recipes.

The outcome is pairing suggestions that achieve better results than ever before. These suggestions can be used to predict relationships between compounds and foods, hinting at new and exciting recipe techniques and driving new perspectives on food science in general.

In this blog, we thought we’d go a little way into the context of the project and of the ingredient pairing science in general!

Sony AI x Korea University

The origins of FlavorGraph lie in the work of Donghyeon Park, Keonwoo Kim and Professor Jaewoo Kang. From November 2019 to March 2020, Donghyeon joined Sony AI as an intern, and during his stay worked on AI for recipe creation. The aim was to develop an intelligent deep learning model that recommends complementary and novel ingredient pairings to help chefs with new creations.

After returning to Korea University, Donghyeon and his team’s research culminated in January 2021 with the publishing of a paper in Scientific Reports, a sister journal of Nature, titled, FlavorGraph: a large-scale food-chemical graph for generating food representations and recommending food pairings.

Our COO, Michael Spranger, described the achievement as follows:

"Ingredient pairing is an excellent challenge for AI systems. It requires AI systems to build models of how people pair ingredients from recipes as well as through other strategies. This work is a great step in AI for creativity within our Gastronomy project."

The context

Chefs traditionally mastered ingredient pairing through their intuition, rather than through scientific knowledge or statistical learning. And we can probably thank the gradual progress of cultural evolution for many of the pairings we see as standard today – cheese and tomato, garlic and ginger, pork and apple.

More recently, science was used to explain why such pairings work. Even novel pairings, like Heston Blumenthal’s caviar and chocolate, were discovered first by him, then later explained using his molecular science. Blumenthal then used this approach to explain and explore how ingredients sharing dominant flavor molecules paired well together.

A 2011 study led by Yong-Yeol Ahn questioned whether this was the only method of reaching ingredient pairing success. Based on analyses of recipes across the world, there was a suggestion that some cultures (especially North American and European) favored recipes that paired ingredients whose flavors matched, but that others (especially East Asian) tended towards recipes whose ingredients’ flavors didn’t match. What has become clear is that ingredient pairing is no simple feat!

The challenge for the team

Based on the complex nature of ingredient pairing and its need to understand food through various scientific attributes, Donghyeon and the team identified two main challenges to overcome in their project:

• How can we utilize available recipe data and meaningful descriptive features (e.g., flavor molecules) to obtain better food representations?
• How can we apply these improved food representations to food pairing?

The solution was FlavorGraph.

FlavorGraph explained

FlavorGraph extracts relationships between food items from a million food recipes, analyzing how chefs and recipe creators have brought ingredients together; and it combines these with information for 1,561 flavor molecules.

Such a graph enables analysis of the chemical and statistical relationships between ingredients, allowing the team to better represent the foods.

Ingredient pairing in the future

As well as relationships between food ingredients and flavor compounds that have not been explored before, the FlavorGraph research will allow greater flexibility for matching single or multiple ingredients to many others. 

As the science develops and we get ever better representations of food, we should discover more and more intriguing pairings of ingredients, as well as new substitutes for ingredients that are either unhealthy or unsustainable.

So we hope that projects like this will continue to complement both the complex ingredient systems fossilized over time through cultural evolution, as well as the electric ingenuity of modern innovators and chefs.

And in addition to individual ingredient pairings, analysis like FlavorGraph will help us to discover new recipes, changing the way we eat forever.

Sony AI is always looking to develop partnerships with academic institutions and food companies to develop knowledge in AI and gastronomy. And we are always looking to work with talented researchers like Donghyeon! Please get in touch if you’d like to discuss more.

More information on FlavorGraph, and the paper, can be found here.