AI Assisted Recipe Creation – Concept Video
Imagining a new dialogue between chefs and AI
August 10, 2021
An integral part of our Gastronomy Flagship Project is imagining the future of AI-led recipe creation. We’ve developed our first concept video for a recipe creation app, bringing to life the different ways in which we imagine chefs can enter a dialogue with AI around dish creation.
Leveraging food data for deliciousness, sustainability and health
Sony AI’s mission is to unleash human imagination and creativity with AI, and one of the ways we envision doing that is through a digital canvas that allows rich conversations between AI systems and chefs, helping them go even further with their creations.
At the center of this work is the belief that AI can assist chefs to achieve deliciousness, sustainability, and health in gastronomy, leveraging the vast amount of food data out there.
Why recipe creation and AI?
Recipe design and creation is a key starting point in a chef’s journey to connect with their diners, and we felt it was vital for us to be there from the very beginning and, if possible, provide a source of inspiration for chefs in their creative process.
We’ve spent the last 15 months engaging with various professionals in the gastronomy space, including chefs and food companies. For instance, our chef interviews can be found on our website, here, with some of the key insights consolidated here. And our research with Korea University to build an ingredient mapping tool called FlavorGraph is presented here.
Throughout this research, chefs told us about the time, energy and resources they invest into delighting customers, and we learnt that...
- They are inspired by numerous sources – nature, art, travel, architecture, music etc.
- They are eager to learn more and have access to wide ranges of exploratory information and science
- They love to work with people who are specialized in different disciplines to identify novel combinations, processes and technologies
- They will try and fail and try again, iterating until they get it right
It seems a perfect challenge for AI! Our goal is to empower chefs with a system that can create meaningful connections across a vast amount of data.
Building our concept
One of the key areas we explored in our concept was the different entry points into recipe design.
We think there are multiple entry points of interest, such as ingredient type, the eater’s health, the seasonality and origin of the ingredients, the availability of certain kitchen equipment...
In our concept video, we focused on starting with an ingredient, imagining a scenario where the chef might come across fresh wasabi while shopping in a market. The AI system identifies the ingredient via a photo the chef takes at the market and presents related facts and information.
From this starting ingredient, the chef might begin exploring possible pairings with other ingredients – for instance as a result of their co-occurrence in recipes, or shared molecules. In our video, they then tweak this based on flavor profile or texture, and then they examine the health and sustainability impact of certain ingredient choices, swapping out ones that suit the body or the planet better. Finally, they look at a more abstract input, exploring how food can evoke the feelings of a piece of jazz.
In such a way, we want to enable multiple routes in and around recipe creation, helping chefs to iterate and build over time.
And to create the narrative, in addition to pulling in the science and AI knowledge of our tech teams, we worked closely with our special adviser, the renowned food expert Francois Chartier, to create an ‘ingredient story’ that produced a delicious result. Among other things, Francois was inspired by the mood and vibration that can form an inspirational bridge between music and food.
Creating a dialogue between chefs and AI
Crucially, in developing our concepts, we want to focus on the idea of a dialogue between chefs and AI. Since we know that recipe creation is an iterative process, we want an interaction model that feels natural for those using it, and that fits into the existing creative processes of chefs. The role of the system is to augment the chef’s creativity, not to provide an absolute answer.
We visualize a tool where chefs can bounce ideas around, learn more about the ingredients they’re targeting, and have a central place for creative discussions with their wider teams.
Our job now is to continue refining this vision of the future, and to continue developing the AI that sits behind it. This is a concept video and, clearly, we are only scratching the surface of what is possible – both from an AI perspective and a recipe creation perspective.
To see how we can develop the vision further and of course make it into a reality, we are always looking for various inputs and partnership opportunities – please get in touch with your ideas!
Sony AI is always looking to develop partnerships with academic institutions and food companies to develop knowledge in AI and gastronomy. And we are always looking to work with talented researchers! Please get in touch if you’d like to discuss more.
Erica Kato Marcus
Director, Strategy & Partnerships
At Sony AI, Erica leads the Strategy & Partnership team that helps define and communicate the company’s strategic direction and develop partnerships that enable Sony AI to realize its mission. She has a broad range of experience working across Corporate and Executive Communications at Sony’s global headquarters in Tokyo, managing a global team of product managers to develop apps and services while at Sony Mobile in Sweden for Xperia smartphones and at Sony Interactive Entertainment for the PlayStation business in the US. She grew up in Japan and earned her BS degree from Boston College, and MBA from IMD Business School in Switzerland. She’s an ardent cook and in her spare time she bakes sourdough bread and is constantly experimenting with new recipes.
Strategy & Partnerships Manager
Born in London but raised in Tokyo, Fred has worked across the world in roles related to product innovation and corporate and business strategy, and he completed his MBA at ESADE in Barcelona in 2019. His work in AI has crossed city simulation, biopharmaceutical drug development and skyscraper energy management. In his role at Sony AI, he is helping set the strategy for the gastronomy project and build relationships with companies across the food industry.
May 11, 2023 | Sony AI
Meet the Team #7: Victoria Matthews, Julienne LaChance, Dora Zhao
AI Ethics sits at the core of our activities at Sony AI. In this installment of our Meet the Team series, meet members of our growing AI Ethics team. This month we feature Victoria…
April 17, 2023 | AI Ethics
Exposing Limitations in Fairness Evaluations: Human Pose Estimation
As AI technology becomes increasingly ubiquitous, we reveal new ways in which AI model biases may harm individuals. In 2018, for example, researchers Joy Buolamwini and Timnit Gebr…
March 16, 2023 | AI Ethics
Being 'Seen' vs. 'Mis-Seen': Tensions Between Privacy and Fairness in Computer V…
Privacy is a fundamental human right that ensures individuals can keep their personal information and activities private. In the context of computer vision, privacy concerns arise …