The Future Kitchen Series – Chef Hajime Yoneda Part 2

In our previous blog, we introduced Hajime Yoneda, the celebrated chef, technologist, artist and philosopher, who is advising us as we explore how AI and robotics can create new value for chefs and the restaurant industry. During our conversations, Chef Yoneda was able to illuminate ways in which the four elements of data, AI, robotics and sensing technologies could come together to be wielded by chefs in the kitchen. In this blog, we explore some of his thoughts on the sensing technologies that can both capture and supply data.

For Yoneda, the vast amount of cooking related information is at present underused and often not even captured. Above all, quality is of course paramount to him, and to achieve this he believes we must understand the ingredients, their state, and the varied effect that cooking will have on them. Critical to the process is the ability to recognize diverse ingredients and prepare them appropriately.

“Every single carrot measures differently in terms of sugar content, water content, fiber and composition. Flavors differ by plot even within a single farm. Consequently, a dish containing ten vegetable varieties and made according to one recipe can taste completely different to the day before, necessitating skilled staff to adjust the flavor balance. But there is no golden rule for this kind of thing.”

Yoneda must understand and account for each ingredient’s variations.

Meanwhile, he is well aware of the complexities and also of the limitations of the kitchen environment. Every kitchen space is different. There are no standards for sink size and location, neither for the placement and variety of equipment. And while chefs must be designers and artists, because of the medium they work in, they lack the luxury of time.

“Unlike production in an industry like fashion, we are in a race against time, with drastic changes in the materials we handle. Unlike fabrics, our ingredients spoil at rapid speeds. Thus, the issue is with how you check and control quality.”

So this raises the questions: how can we make an impact with new sensing technologies? Where should they be applied? How should they be used? Four of the areas Yoneda talked us through show remarkable opportunities for development.

1. Data to support new recipe development Creating a new dish, or a new restaurant experience, can call on every discipline under the sun. A fundamental step is picking ingredients, and here Yoneda sees opportunities for support.

“When determining flavors for the cuisine at HAJIME, we value the opinions of every team member, working together to propose multiple ingredient combinations. It would be incredibly helpful in such situations to be able to tap technology showing optimal ingredient combinations.”

Moreover, Yoneda sees a job for data and sensors in understanding the human brain and responding to the entire experience of eating. Putting it all together can look like the job of an orchestra conductor.

“Much like music, it is essential to modulate when serving food. This is about skillfully inducing fluctuations in the human brain’s emotional responses. Pleasure and discomfort. Dopamine and beta-endorphins. Temperature range, texture, flavor contrast, tableware size, horizontal and vertical dimensions, how well it melts in the mouth, ease of chewing, length of aftertaste, if it’s liquid or otherwise… being aware of such parameters, and creating contrasts, enables us to create recipes and services that are three-dimensional in nature, like a symphony.”

At this critical stage of designing the experience, getting to grips with all this data to aid a chef’s thinking seems significant.

Yoneda’s sketchbook, where he plans new dish concepts.

2. Sensing to analyze and capture cooking When a chef creates a new dish, it’s rarely a linear process like following a mathematical equation. It’s one of trial, error and experimentation – and when something goes perfectly, it’s crucial to know what led to the success. Small changes can make big differences.

For this reason, Yoneda creates highly specific recipes, controlling contents down to the millimeter, milligram and single degree in temperature, to guarantee the quality of flavors served at the restaurant. But while it is possible to quantify volumes, time or temperature during cooking, there are countless elements in the process that cannot yet be quantified.

“Digitizing the data on flavor and variables (air temperature, humidity, time of day, ingredients’ internal changes and so on), as well as on the values that determine the ultimate flavors, should make it possible to recreate those flavors. Using AI to analyze the data from leading chefs and verbalizing the not yet quantified or verbalized parts could even result in anyone being able to recreate a certain chef’s flavors.”

By capturing the data of a chef’s cooking, Yoneda believes the seemingly miraculous creations could be recreated and replicated, time and again.

Bouillon at HAJIME.

3. Sensing in refrigerators to monitor ingredients Yoneda also spoke about potential improvements to refrigerators used to store ingredients. HAJIME uses four refrigerators with different temperature settings to manage food, but they provide no data on what is in each fridge or when it was purchased.

“I think data management could facilitate notifications on best-before and expiration dates and enable us to buy only what we need and to use it all, leading to better prevention of food loss and waste.”

The reduction of food waste is a major area of exploration in the industry, and Yoneda is keen to support it. Rich data on what is happening to the ingredients inside the fridge – for example measuring ethylene gas emitted by different produce – could also allow the fridge to adjust itself, reducing the pressure on kitchen staff.

One of the refrigerators at HAJIME – sensing technology in cold storage like this could lower food waste.

4. Data enabling individual customer responses Of course with a great sensitivity to privacy, HAJIME gathers information from customers to tailor the experience to them as much as possible. This information can then be used each time the customer visits the restaurant, so any enhancement of this data is valuable to the Hajime team.

“Be it online or over the phone, it starts from the very first point of contact. A guest may be celebrating a birthday, and there may be something they really want to eat or prefer not to – all this information is shared with the kitchen. Are they left-handed or right-handed? How do they use a knife and fork? Right hand for a spoon but opposite for the fork? Where do they typically place their wine glass? We digitize all that information.”

Yoneda is also fascinated by data that – as long as the correct privacy protections are in place – can be obtained in real time during a meal, to boost the satisfaction of the eater.

“Sensing technologies and AI have the potential to be better at capturing exactly what guests want, by reading minute facial impressions, for example. They may be saying it’s delicious, but not really thinking that. Sensing and analyzing living data from guests on gaze, mimetic muscles, perspiration, body temperature and other factors could help us understand their true state and feelings, and unconscious desires. We could contemplate systems that provide real-time feedback on service and seasoning.”

This highly personalized understanding of, and response to, an eater is a prime area for sensing technology to target.

Components for HAJIME dishes – customer preferences are ‘baked in’ to dish preparation.

Such opportunities for sensing and data appear significant, and Yoneda is eager to see their progress in the industry. In a highly complex environment, one that lacks the consistency and flexibility of so many other industries, he can see how the opportunities could reshape restaurants for the better.

In our next blog, we explore Yoneda’s thoughts on how robots could make a similar impact in the kitchen, delving into the use cases they can be applied to, and how this could benefit chefs.

Erica Kato Marcus

Director, Strategy & Partnerships

At Sony AI, Erica leads the Strategy & Partnership team that helps define and communicate the company’s strategic direction and develop partnerships that enable Sony AI to realize its mission. She has a broad range of experience working across Corporate and Executive Communications at Sony’s global headquarters in Tokyo, managing a global team of product managers to develop apps and services while at Sony Mobile in Sweden for Xperia smartphones and at Sony Interactive Entertainment for the PlayStation business in the US. She grew up in Japan and earned her BS degree from Boston College, and MBA from IMD Business School in Switzerland. She’s an ardent cook and in her spare time she bakes sourdough bread and is constantly experimenting with new recipes.

Fred Gifford

Strategy & Partnerships Manager

Born in London but raised in Tokyo, Fred has worked across the world in roles related to product innovation and corporate and business strategy, and he completed his MBA at ESADE in Barcelona in 2019. His work in AI has crossed city simulation, biopharmaceutical drug development and skyscraper energy management. In his role at Sony AI, he is helping set the strategy for the gastronomy project and build relationships with companies across the food industry.