Expert Human-Level Driving in Gran Turismo Sport Using Deep Reinforcement Learning with Image-based Representation

Ryuji Imamura

Takuma Seno

Kenta Kawamoto

Michael Spranger

NeurIPS-2021, Deep RL Workshop



When humans play virtual racing games, they use visual environmental information on the game screen to understand the rules within the environments. In contrast, a state-of-the-art realistic racing game AI agent that outperforms human players does not use image-based environmental information but the compact and precise measurements provided by the environment. In this paper, a vision-based control algorithm is proposed and compared with human player performances under the same conditions in realistic racing scenarios using Gran Turismo Sport (GTS), which is known as a high-fidelity realistic racing simulator. In the proposed method, the environmental information that constitutes part of the observations in conventional state-of-the-art methods is replaced with feature representations extracted from game screen images. We demonstrate that the proposed method performs expert human-level vehicle control under high-speed driving scenarios even with game screen images as high-dimensional inputs. Additionally, it outperforms the built-in AI in GTS in a time trial task, and its score places it among the top 10% approximately 28,000 human players.

Related Publications

Skill-Critic: Refining Learned Skills for Hierarchical Reinforcement Learning

RAL, 2024
Ce Hao*, Catherine Weaver*, Chen Tang*, Kenta Kawamoto, Masayoshi Tomizuka*, Wei Zhan*

Hierarchical reinforcement learning (RL) can accelerate long-horizon decision-making by temporally abstracting a policy into multiple levels. Promising results in sparse reward environments have been seen with skills , i.e. sequences of primitive actions. Typically, a skill …

Model-based Reinforcement Learning with Scalable Composite Policy Gradient Estimators

ICML, 2023
Paavo Parmas*, Takuma Seno, Yuma Aoki*

In model-based reinforcement learning (MBRL), policy gradients can be estimated either by derivative-free RL methods, such as likelihood ratio gradients (LR), or by backpropagating through a differentiable model via reparameterization gradients (RP). Instead of using one or …

Improving Artificial Intelligence with Games

Science, 2023
Peter R. Wurman, Peter Stone, Michael Spranger

Games continue to drive progress in the development of artificial intelligence.

  • HOME
  • Publications
  • Expert Human-Level Driving in Gran Turismo Sport Using Deep Reinforcement Learning with Image-based Representation


Shape the Future of AI with Sony AI

We want to hear from those of you who have a strong desire
to shape the future of AI.