Authors
- Pete Wurman
- Samuel Barrett
- Kenta Kawamoto
- James MacGlashan
- Kaushik Subramanian
- Thomas J. Walsh
- Roberto Capobianco
- Alisa Devlic
- Franziska Eckert
- Florian Fuchs
- Leilani Gilpin
- Piyush Khandelwal
- Varun Kompella
- Hao Chih Lin
- Patrick MacAlpine
- Declan Oller
- Takuma Seno
- Craig Sherstan
- Michael D. Thomure
- Houmehr Aghabozorgi
- Leon Barrett
- Rory Douglas
- Dion Whitehead Amago
- Peter Dürr
- Peter Stone
- Michael Spranger
- Hiroaki Kitano
Venue
- Nature
Date
- 2022
Share
Outracing Champion Gran Turismo Drivers with Deep Reinforcement Learning
Franziska Eckert
Leilani Gilpin
Hao Chih Lin
Dion Whitehead Amago
Nature
2022
Abstract
Many potential applications of artificial intelligence involve making real-time decisions in physical systems while interacting with humans. Automobile racing represents an extreme example of these conditions; drivers must execute complex tactical manoeuvres to pass or block opponents while operating their vehicles at their traction limits. Racing simulations, such as the PlayStation game Gran Turismo, faithfully reproduce the non-linear control challenges of real race cars while also encapsulating the complex multi-agent interactions. Here we describe how we trained agents for Gran Turismo that can compete with the world’s best e-sports drivers. We combine state-of-the-art, model-free, deep reinforcement learning algorithms with mixed-scenario training to learn an integrated control policy that combines exceptional speed with impressive tactics. In addition, we construct a reward function that enables the agent to be competitive while adhering to racing’s important, but under-specified, sportsmanship rules. We demonstrate the capabilities of our agent, Gran Turismo Sophy, by winning a head-to-head competition against four of the world’s best Gran Turismo drivers. By describing how we trained championship-level racers, we demonstrate the possibilities and challenges of using these techniques to control complex dynamical systems in domains where agents must respect imprecisely defined human norms.
Related Publications
Existing collaborative self-supervised learning (SSL) schemes are not suitable for cross-client applications because of their expensive computation and large local data requirements. To address these issues, we propose MocoSFL, a collaborative SSL framework based on Split Fe…
Continual Test-time Adaptation (CTA) is a promising art to secure accuracy gains in continually-changing environments. The state-of-the-art adaptations improve out-of-distribution model accuracy via computation-efficient online test-time gradient descents but meanwhile cost …
Despite the advancement of machine learning techniques in recent years, state-of-the-art systems lack robustness to “real world” events, where the input distributions and tasks encountered by the deployed systems will not be limited to the original training context, and syst…
JOIN US
Shape the Future of AI with Sony AI
We want to hear from those of you who have a strong desire
to shape the future of AI.
