Meet the Team #6: Florian Fuchs, Takuma Seno, Yunshu Du
Life at Sony AI
November 1, 2022
The sixth installment of our Meet the Team series features members of the global Sony AI team who contributed to the groundbreaking research, Outracing Champion Gran Turismo Drivers with Deep Reinforcement Learning, and created the AI racing agent Gran Turismo Sophy, announced in February. This month we feature Florian, Takuma, and Yunshu, who collaborated from their respective locations of work in Zurich, Tokyo and the U.S., to develop and train Gran Turismo Sophy into a champion race car driver. You can see their work in action in our new docuseries, "The Team Behind GT Sophy," which offers a behind-the-scenes look at the development of GT Sophy and the team who created it.
Florian Fuchs
AI Engineer
Based In
Zurich, Switzerland
Areas of Speciality
Reinforcement Learning
![](/uploads/Florian%20Fuchs.png)
What made you interested in working in AI and your area of expertise?
The concept of training an AI agent from scratch without human knowledge explicitly encoded in its behavior truly
fascinated me. In the spring of 2019, I had the chance to collaborate with Sony for my master’s thesis on the topic of
reinforcement learning for racing in the Gran Turismo (GT) Sport simulation. I always loved video games, so taking up
this challenge of using reinforcement learning to enhance video games was only natural to me. Since joining Sony AI, I
have had the opportunity to collaborate with talented researchers from all over the world. Joining forces with them to
drive progress in this rapidly growing domain is as challenging as rewarding.
What made you excited about joining Sony AI and what do you hope to accomplish?
I joined the gaming flagship project of Sony AI in August 2020 after finishing my work on autonomous racing in Gran
Turismo Sport at Sony R&D Center. I was super excited to join a team of experienced Reinforcement Learning researchers
to help transfer our initial proof of concept on Reinforcement Learning for autonomous racing in GT into what has now
become Gran Turismo (GT) Sophy™.
My current focus lies on applying reinforcement learning algorithms to the domain of games to make them more exciting. My personal goal is to integrate a deep reinforcement learning based model into a commercial game. While recent work on deep reinforcement learning showed great success in the domain of games, the results didn’t make it into a publicly available product. I’m motivated to show that this is possible.
What are your thoughts about the future of AI and AI for creativity?
I believe that in the near future AI will be even more present in our lives. Recent breakthroughs in areas such as
reinforcement learning, computer vision, robotic control and natural language processing will be ubiquitous in our lives
through a vast range of applications, many yet to be discovered. Therefore, I believe that it is crucial that future AI
applications are designed to support and improve our everyday tasks and activities such that they are beneficial to
everyone's life. I believe that an important step towards this is the democratization of machine learning. This is for
example taking place in the form of researchers sharing their findings as well as their machine learning models and code
with the public. Distributing the knowledge to train machine learning algorithms is an important factor for increasing
diversity in the field.
More generally, I see AI as a tool to assist humans, not replace them. In the near future, AI tools will allow us to move away from repetitive and dangerous tasks and instead focus on the innovative and creative aspects.
Florian is an AI engineer for the Game AI flagship project at Sony AI. He obtained an MSc in computer science with a focus on machine learning at the University of Zurich. For his master's thesis he applied reinforcement learning on the challenge of autonomous racing in the Gran Turismo Sport simulation in cooperation with Sony R&D Center and the Robotics and Perception Group at UZH/ETHZ. The approach achieved superhuman time trial performance, outperforming all of over 50,000 human competitors. During his master studies he was a visiting student at ETHZ and SNU. Before that, Florian acquired a BSc in information systems. His thesis on analyzing the environmental impact caused by media consumption was awarded with the Environmental Informatics student prize at EnviroInfo 2016.
Takuma Seno
AI Engineer
Based In
Tokyo, Japan
Areas of Speciality
Deep Reinforcement Learning
![](/uploads/meet6-seno-takuma.jpg)
What made you interested in working in AI or your area of expertise?
I believe that AI, especially deep reinforcement learning (deep RL), has an unprecedented ability to enrich our
creativity. Deep RL is the way to let AI train by itself just as we demonstrated in the Gran Turismo (GT) Sophy™
project. GT Sophy demonstrates fantastic maneuvers without learning from explicit human knowledge. From a technical
point of view, this achievement suggests that AI can find more complex algorithms than humans can possibly design.
However, the more excellent finding we made in this project was that all professional humans are inspired by the tactics
of GT Sophy. This means that GT Sophy is not just a technical achievement, but also demonstrates the potential of the
future of AI that enhances human creativity and imagination. There are many more places where we can apply deep RL in
Sony because of a wide variety of business domains. I’m very excited to see how AI inspires us next.
What made you excited about joining Sony AI and what do you hope to accomplish?
Before I joined Sony AI, I was working for the Gran Turismo project at the Sony R&D Center as a part-time researcher. I
was very excited to see the GT Sophy project become one of Sony AI’s flagship projects and for the research to achieve
the cover of Nature.
Sony AI is a place where you can work on ground-breaking projects. I am currently working with the Game AI flagship
project where we are tackling many practical and theoretical reinforcement learning challenges. There are an enormous
number of potential projects where we can leverage the power of reinforcement learning at Sony, and I'm very excited to
see what we can do in the future.
What are your thoughts about AI for creativity?
Future AI will learn without the need for a dataset. It will learn everything by itself and truly understand us and help
us to create something new just like what we're currently doing at Sony AI. Gran Turismo Sophy will be a super important
AI project because it is the only AI that is designed to play alongside humans while competing with each other. This
technology will be essential when we build AI and robots that share the living space with us.
Takuma joined Sony AI in Oct 2020 as a machine learning engineer, following part-time research positions at Sony R&D Center, Ghelia and Okinawa Institute of Science and Technology. He received his master’s degree in computer science at Keio University in 2019, and is currently pursuing his Ph.D. Takuma’s main research interest is deep reinforcement learning. He developed an offline deep reinforcement learning library, d3rlpy, funded by the IPA MITOU program in 2020, and was certified as a MITOU Super Creator in 2021.
Yunshu Du
Research Scientist
Based In
Denver, Colorado, U.S.
Areas of Speciality
Reinforcement Learning
![](/uploads/Yunshu%20Du.jpg)
What made you interested in working in AI and your area of expertise?
I am generally curious about how AI systems work behind the scenes. I actually had no experience with AI prior to
graduate school. Perhaps like many who do not have much knowledge of AI, my impression of AI for a long time was,
"SkyNet will destroy us!" I took my first AI class in graduate school and there I learned that an agent's learning
process is controlled by algorithms. I started to wonder, "What kind of algorithm would be needed if we were to train a
terminator?" Of course, I know now that this is an extremely challenging problem, and to date, we are still unable to
solve it. But that curiosity was the beginning of my AI journey. During my Ph.D. study, I was very lucky to experience
several breakthroughs of new AI technologies, which encouraged me to stay curious and dig deeper into their
behind-the-scenes mechanisms.
What made you excited about joining Sony AI and what do you hope to accomplish?
To me, the most exciting things about joining Sony AI are the people and the mission. Back while I was still a Ph.D.
candidate, I had read many research papers written by individuals who are now my colleagues. It is very exciting to work
side-by-side with a group of such talented and knowledgeable people. Sony AI's mission is to build AI systems that can
enhance human imagination and creativity, and I believe in the idea of building AI agents that can work together
with--not replace--humans. I look forward to working with my Sony AI colleagues to develop real-world AI applications
that can bring people joy.
If you had to give one piece of advice to an individual starting a career in AI, what would it be?
Be patient and stay positive. Despite how “cool” people may think AI is, the process of developing an AI system from
scratch takes a lot of time as well as many failures. You might feel frustrated because things don't work -- and this is
completely normal. Give yourself a big smile and keep going. Things will always work out in the end!
Yunshu joined Sony AI as a research scientist in 2021 after receiving her Ph.D. in Computer Science from Washington State University. Her research interests include Reinforcement Learning (RL), RL for real-world applications, and RL with human interactions.
Latest Blog
![](/assets_c/2024/08/SonyAI_SightsonAI_YukiMitsufuji_BlogHeader_1080x1080_V2_AM_070324-thumb-600x600-767.png)
August 14, 2024 | Sony AI
Sights on AI: Yuki Mitsufuji Shares Inspiration for AI Research into Music and S…
The Sony AI team is a diverse group of individuals working to accomplish one common goal: accelerate the fundamental research and development of AI and enhance human imagination an…
![](/assets_c/2024/08/SonyAI_ReinforcementLearningConference_BlogHeader_1080x1080_v3-thumb-600x600-771.png)
August 10, 2024 | Game AI
Sony AI at the Reinforcement Learning Conference 2024
Sony AI will be participating in the Reinforcement Learning (RL) Conference in Amherst, Massachusetts, from August 9 to 12, 2024 where we will be joining some of the brightest mind…
![](/assets_c/2024/07/SonyAI_ICMLBestPaper_Blog_1080x1080_v3-thumb-600x600-755.png)
July 27, 2024 | AI Ethics
Ushering in Needed Change in the Pursuit of More Diverse Datasets
Sony AI, Research Scientist, Jerone Andrews’ paper, "Measure Dataset Diversity, Don't Just Claim It", has won a Best Paper Award at ICML 2024. This recognition is a testament to th…