AI Assisted Recipe Creation – Concept Video

An integral part of our Gastronomy Flagship Project is imagining the future of AI-led recipe creation. We’ve developed our first concept video for a recipe creation app, bringing to life the different ways in which we imagine chefs can enter a dialogue with AI around dish creation.

Leveraging food data for deliciousness, sustainability and health

Sony AI’s mission is to unleash human imagination and creativity with AI, and one of the ways we envision doing that is through a digital canvas that allows rich conversations between AI systems and chefs, helping them go even further with their creations.

At the center of this work is the belief that AI can assist chefs to achieve deliciousness, sustainability, and health in gastronomy, leveraging the vast amount of food data out there.

Why recipe creation and AI?

Recipe design and creation is a key starting point in a chef’s journey to connect with their diners, and we felt it was vital for us to be there from the very beginning and, if possible, provide a source of inspiration for chefs in their creative process.

We’ve spent the last 15 months engaging with various professionals in the gastronomy space, including chefs and food companies. For instance, our chef interviews can be found on our website, here, with some of the key insights consolidated here. And our research with Korea University to build an ingredient mapping tool called FlavorGraph is presented here.

Throughout this research, chefs told us about the time, energy and resources they invest into delighting customers, and we learnt that...

• They are inspired by numerous sources – nature, art, travel, architecture, music etc.
• They are eager to learn more and have access to wide ranges of exploratory information and science
• They love to work with people who are specialized in different disciplines to identify novel combinations, processes and technologies
• They will try and fail and try again, iterating until they get it right

It seems a perfect challenge for AI! Our goal is to empower chefs with a system that can create meaningful connections across a vast amount of data.

Building our concept

One of the key areas we explored in our concept was the different entry points into recipe design.

We think there are multiple entry points of interest, such as ingredient type, the eater’s health, the seasonality and origin of the ingredients, the availability of certain kitchen equipment...

In our concept video, we focused on starting with an ingredient, imagining a scenario where the chef might come across fresh wasabi while shopping in a market. The AI system identifies the ingredient via a photo the chef takes at the market and presents related facts and information.

From this starting ingredient, the chef might begin exploring possible pairings with other ingredients – for instance as a result of their co-occurrence in recipes, or shared molecules. In our video, they then tweak this based on flavor profile or texture, and then they examine the health and sustainability impact of certain ingredient choices, swapping out ones that suit the body or the planet better. Finally, they look at a more abstract input, exploring how food can evoke the feelings of a piece of jazz.

In such a way, we want to enable multiple routes in and around recipe creation, helping chefs to iterate and build over time.

And to create the narrative, in addition to pulling in the science and AI knowledge of our tech teams, we worked closely with our special adviser, the renowned food expert Francois Chartier, to create an ‘ingredient story’ that produced a delicious result. Among other things, Francois was inspired by the mood and vibration that can form an inspirational bridge between music and food.

Creating a dialogue between chefs and AI

Crucially, in developing our concepts, we want to focus on the idea of a dialogue between chefs and AI. Since we know that recipe creation is an iterative process, we want an interaction model that feels natural for those using it, and that fits into the existing creative processes of chefs. The role of the system is to augment the chef’s creativity, not to provide an absolute answer.

We visualize a tool where chefs can bounce ideas around, learn more about the ingredients they’re targeting, and have a central place for creative discussions with their wider teams.

The Future

Our job now is to continue refining this vision of the future, and to continue developing the AI that sits behind it. This is a concept video and, clearly, we are only scratching the surface of what is possible – both from an AI perspective and a recipe creation perspective.

To see how we can develop the vision further and of course make it into a reality, we are always looking for various inputs and partnership opportunities – please get in touch with your ideas!

Sony AI is always looking to develop partnerships with academic institutions and food companies to develop knowledge in AI and gastronomy. And we are always looking to work with talented researchers! Please get in touch if you’d like to discuss more.