Thomas Walsh
Profile
Thomas is a senior research scientist at Sony AI where he investigates the use of reinforcement learning in game-AI applications. He received his Ph.D. in computer science from Rutgers University and a B.S. in computer science from UMBC. Before joining Sony AI, Tom led an industry AI team focused on workforce applications and held research positions at MIT, the University of Kansas, and the University of Arizona. Tom’s previous research spans multiple domains including robotics, education, and logistics. His work has been published in top AI conferences and journals including AAAI, ICML, and NeurIPS.
Publications
Event Tables for Efficient Experience Replay
COLLAS, 2023 | Varun Kompella, Thomas Walsh, Samuel Barrett, Peter R. Wurman, Peter Stone
Experience replay (ER) is a crucial component of many deep reinforcement learning (RL) systems. However, uniform sampling from an ER buffer can lead to slow convergence and unstable asymptotic behaviors. This paper introduces Stratified Sampling from Event Tables (SSET), whi...
Composing Efficient, Robust Tests for Policy Selection
UAI, 2023 | Dustin Morrill, Thomas Walsh, Daniel Hernandez, Peter R. Wurman, Peter Stone
Modern reinforcement learning systems produce many high-quality policies throughout the learning process. However, to choose which policy to actually deploy in the real world, they must be tested under an intractable number of environmental conditions. We introduce RPOSST, a...
Outracing Champion Gran Turismo Drivers with Deep Reinforcement Learning
NATURE, 2022 | Pete Wurman, Samuel Barrett, Kenta Kawamoto, James MacGlashan, Kaushik Subramanian, Thomas Walsh, Roberto Capobianco, Alisa Devlic, Franziska Eckert, Florian Fuchs, Leilani Gilpin, Piyush Khandelwal, Varun Kompella, Hao Chih Lin, Patrick MacAlpine, Declan Oller, Takuma Seno, Craig Sherstan, Michael D. Thomure, Houmehr Aghabozorgi, Leon Barrett, Rory Douglas, Dion Whitehead Amago, Peter Dürr, Peter Stone, Michael Spranger, Hiroaki Kitano
Many potential applications of artificial intelligence involve making real-time decisions in physical systems while interacting with humans. Automobile racing represents an extreme example of these conditions; drivers must execute complex tactical manoeuvres to pass or block...
Blog Posts
Event Tables for Efficient Experience Replay
December 14, 2023 | Peter Stone, Pete Wurman, Game AI, GT Sophy, Thomas Walsh, Samuel Barrett, Varun Kompella
Each of us carries a core set of experiences, events that stand out as particularly important and have shaped our lives more than an average day. ...
RPOSST: Testing an AI Agent for Deployment in the Real World
December 5, 2023 | Peter Stone, Pete Wurman, Game AI, Thomas Walsh, Dustin Morrill
Bleary-eyed engineers know the anxiety that comes with a deployment, and the importance of testing every aspect of a product before it goes to the ...
The Race to Turn a World-class AI into a World Champion
October 4, 2022 | Game AI, GT Sophy, Thomas Walsh, James MacGlashan, Kaushik Subramanian
GT SOPHY TECHNICAL SERIES Starting in 2020, the research and engineering team at Sony AI set out to do something that had never been done before: ...
How to Train Your Race Car
July 12, 2022 | Gaming, GT Sophy, Thomas Walsh, Craig Sherstan, Varun Kompella
GT SOPHY TECHNICAL SERIES Starting in 2020, the research and engineering team at Sony AI set out to do something that had never been done before: ...
Meet the Team #4: Kenta, Alisa and Thomas
April 4, 2022 | Life at Sony AI, Thomas Walsh, Kenta Kawamoto, Alisa Devlic
The next installments of our Meet the Team series will feature members of the global Sony AI team who contributed to the groundbreaking research, ...