Meet the Team #6: Florian Fuchs, Takuma Seno, Yunshu Du

The sixth installment of our Meet the Team series features members of the global Sony AI team who contributed to the groundbreaking research, Outracing Champion Gran Turismo Drivers with Deep Reinforcement Learning, and created the AI racing agent Gran Turismo Sophy, announced in February. This month we feature Florian, Takuma, and Yunshu, who collaborated from their respective locations of work in Zurich, Tokyo and the U.S., to develop and train Gran Turismo Sophy into a champion race car driver. You can see their work in action in our new docuseries, "The Team Behind GT Sophy," which offers a behind-the-scenes look at the development of GT Sophy and the team who created it.

Florian Fuchs

AI Engineer

Based In

Zurich, Switzerland

Areas of Speciality

Reinforcement Learning

What made you interested in working in AI and your area of expertise?
The concept of training an AI agent from scratch without human knowledge explicitly encoded in its behavior truly fascinated me. In the spring of 2019, I had the chance to collaborate with Sony for my master’s thesis on the topic of reinforcement learning for racing in the Gran Turismo (GT) Sport simulation. I always loved video games, so taking up this challenge of using reinforcement learning to enhance video games was only natural to me. Since joining Sony AI, I have had the opportunity to collaborate with talented researchers from all over the world. Joining forces with them to drive progress in this rapidly growing domain is as challenging as rewarding.

What made you excited about joining Sony AI and what do you hope to accomplish?
I joined the gaming flagship project of Sony AI in August 2020 after finishing my work on autonomous racing in Gran Turismo Sport at Sony R&D Center. I was super excited to join a team of experienced Reinforcement Learning researchers to help transfer our initial proof of concept on Reinforcement Learning for autonomous racing in GT into what has now become Gran Turismo (GT) Sophy™.

My current focus lies on applying reinforcement learning algorithms to the domain of games to make them more exciting. My personal goal is to integrate a deep reinforcement learning based model into a commercial game. While recent work on deep reinforcement learning showed great success in the domain of games, the results didn’t make it into a publicly available product. I’m motivated to show that this is possible.

What are your thoughts about the future of AI and AI for creativity?
I believe that in the near future AI will be even more present in our lives. Recent breakthroughs in areas such as reinforcement learning, computer vision, robotic control and natural language processing will be ubiquitous in our lives through a vast range of applications, many yet to be discovered. Therefore, I believe that it is crucial that future AI applications are designed to support and improve our everyday tasks and activities such that they are beneficial to everyone's life. I believe that an important step towards this is the democratization of machine learning. This is for example taking place in the form of researchers sharing their findings as well as their machine learning models and code with the public. Distributing the knowledge to train machine learning algorithms is an important factor for increasing diversity in the field.

More generally, I see AI as a tool to assist humans, not replace them. In the near future, AI tools will allow us to move away from repetitive and dangerous tasks and instead focus on the innovative and creative aspects.

Florian is an AI engineer for the Game AI flagship project at Sony AI. He obtained an MSc in computer science with a focus on machine learning at the University of Zurich. For his master's thesis he applied reinforcement learning on the challenge of autonomous racing in the Gran Turismo Sport simulation in cooperation with Sony R&D Center and the Robotics and Perception Group at UZH/ETHZ. The approach achieved superhuman time trial performance, outperforming all of over 50,000 human competitors. During his master studies he was a visiting student at ETHZ and SNU. Before that, Florian acquired a BSc in information systems. His thesis on analyzing the environmental impact caused by media consumption was awarded with the Environmental Informatics student prize at EnviroInfo 2016.

Takuma Seno

AI Engineer

Based In

Tokyo, Japan

Areas of Speciality

Deep Reinforcement Learning

What made you interested in working in AI or your area of expertise?
I believe that AI, especially deep reinforcement learning (deep RL), has an unprecedented ability to enrich our creativity. Deep RL is the way to let AI train by itself just as we demonstrated in the Gran Turismo (GT) Sophy™ project. GT Sophy demonstrates fantastic maneuvers without learning from explicit human knowledge. From a technical point of view, this achievement suggests that AI can find more complex algorithms than humans can possibly design. However, the more excellent finding we made in this project was that all professional humans are inspired by the tactics of GT Sophy. This means that GT Sophy is not just a technical achievement, but also demonstrates the potential of the future of AI that enhances human creativity and imagination. There are many more places where we can apply deep RL in Sony because of a wide variety of business domains. I’m very excited to see how AI inspires us next.

What made you excited about joining Sony AI and what do you hope to accomplish?
Before I joined Sony AI, I was working for the Gran Turismo project at the Sony R&D Center as a part-time researcher. I was very excited to see the GT Sophy project become one of Sony AI’s flagship projects and for the research to achieve the cover of Nature. Sony AI is a place where you can work on ground-breaking projects. I am currently working with the Game AI flagship project where we are tackling many practical and theoretical reinforcement learning challenges. There are an enormous number of potential projects where we can leverage the power of reinforcement learning at Sony, and I'm very excited to see what we can do in the future.

What are your thoughts about AI for creativity?
Future AI will learn without the need for a dataset. It will learn everything by itself and truly understand us and help us to create something new just like what we're currently doing at Sony AI. Gran Turismo Sophy will be a super important AI project because it is the only AI that is designed to play alongside humans while competing with each other. This technology will be essential when we build AI and robots that share the living space with us.

Takuma joined Sony AI in Oct 2020 as a machine learning engineer, following part-time research positions at Sony R&D Center, Ghelia and Okinawa Institute of Science and Technology. He received his master’s degree in computer science at Keio University in 2019, and is currently pursuing his Ph.D. Takuma’s main research interest is deep reinforcement learning. He developed an offline deep reinforcement learning library, d3rlpy, funded by the IPA MITOU program in 2020, and was certified as a MITOU Super Creator in 2021.

Yunshu Du

Research Scientist

Based In

Denver, Colorado, U.S.

Areas of Speciality

Reinforcement Learning

What made you interested in working in AI and your area of expertise?
I am generally curious about how AI systems work behind the scenes. I actually had no experience with AI prior to graduate school. Perhaps like many who do not have much knowledge of AI, my impression of AI for a long time was, "SkyNet will destroy us!" I took my first AI class in graduate school and there I learned that an agent's learning process is controlled by algorithms. I started to wonder, "What kind of algorithm would be needed if we were to train a terminator?" Of course, I know now that this is an extremely challenging problem, and to date, we are still unable to solve it. But that curiosity was the beginning of my AI journey. During my Ph.D. study, I was very lucky to experience several breakthroughs of new AI technologies, which encouraged me to stay curious and dig deeper into their behind-the-scenes mechanisms.

What made you excited about joining Sony AI and what do you hope to accomplish?
To me, the most exciting things about joining Sony AI are the people and the mission. Back while I was still a Ph.D. candidate, I had read many research papers written by individuals who are now my colleagues. It is very exciting to work side-by-side with a group of such talented and knowledgeable people. Sony AI's mission is to build AI systems that can enhance human imagination and creativity, and I believe in the idea of building AI agents that can work together with--not replace--humans. I look forward to working with my Sony AI colleagues to develop real-world AI applications that can bring people joy.

If you had to give one piece of advice to an individual starting a career in AI, what would it be?
Be patient and stay positive. Despite how “cool” people may think AI is, the process of developing an AI system from scratch takes a lot of time as well as many failures. You might feel frustrated because things don't work -- and this is completely normal. Give yourself a big smile and keep going. Things will always work out in the end!

Yunshu joined Sony AI as a research scientist in 2021 after receiving her Ph.D. in Computer Science from Washington State University. Her research interests include Reinforcement Learning (RL), RL for real-world applications, and RL with human interactions.