Meet the Team #5: Craig, Piyush, and Samuel

The fifth installment of our Meet the Team series features members of the global Sony AI team who contributed to the groundbreaking research, Outracing Champion Gran Turismo Drivers with Deep Reinforcement Learning, and created the AI racing agent Gran Turismo Sophy™, announced in February. This month we feature Craig, Piyush, and Samuel, whose interests in science fiction, robots and artificial intelligence from a young age inspired them to pursue research in reinforcement learning and eventually to join Sony AI where they are part of our Game AI team.

Craig Sherstan

Research Scientist

Based In

Edmonton, Alberta, Canada

Areas of Expertise

Reinforcement Learning

What made you interested in working in AI and your area of expertise?
I've been engrossed with science fiction since I was very young. I love imagining how science can change the world and even what it means to be human. Now I'm working towards translating many of those fantastic ideas from sci-fi into reality.

I'm particularly interested in AI and Reinforcement Learning because these technologies can potentially have such a huge positive transformative effect on society. AI will allow us to do and achieve things that we've never been able to before. One thing I'm really interested in is expanding the cognitive, and even moral, capabilities of humans themselves, and I see the synergy of AI and humans as one of the ways towards that.

What made you excited about joining Sony AI and what do you hope to accomplish?
Sony AI has a great team of scientists, many of whom I previously had the pleasure of working with as an intern. I look forward to finding new ways for Sony AI to really take advantage of the unique opportunities that we have in being part of the larger Sony group. In particular, I want to work on projects that enhance human creativity and abilities - think AI helping humans. Sony AI is still relatively new and small and I think that provides an incredible opportunity to shape the future direction of the company. I'm excited to be part of that.

What are your thoughts about the future of AI and AI for creativity?
Obviously, there are many opportunities for the application of AI including health care, automation, and even scientific discovery; another is creativity. One way to value creativity is for its output. That is, we might value creativity because we can consume the output such as a movie or music. Another way to value creativity is for the experience of creating itself - the pleasure one can have from being in that creative process. I think there is a really cool opportunity for AI to come alongside humans and facilitate that process. For example, singing is a relatively easy and immediate way for us to convert musical thoughts and ideas into song. Could we use AI to make the process of composing a song - its melodies, percussion, dynamic movement and even instrument voices - as intuitive as singing? Could an AI help us translate what we imagine in our minds into the real-world?

I'd also really like to see how we can use AI to help us, as individual humans, become who we want to be. Could an AI help people emotionally? Could an AI help people overcome depression or deal with trauma? Could an AI help us to focus or avoid procrastination? Could an AI help us to be more compassionate or kind?

After a career in software development, Craig returned to the University of Alberta to study artificial intelligence, particularly the field of reinforcement learning. After completing his master’s degree he was awarded the Vanier scholarship, which enabled Craig to pursue a doctoral degree while raising his two young kids alongside his wife. The primary focus of his doctoral research was on the use of prediction for perception in robots. That is, robots that represent their world and their interactions with it by making predictions about their own sensory experiences.

Piyush Khandelwal

Senior Research Scientist

Based In

Austin, Texas

Areas of Expertise

Robotics, Applied Reinforcement Learning

What made you interested in working in AI or your area of expertise?
I've always had a lot of interest in figuring out how things worked so that I can learn how to improve them. My interests included computer science, electronics, and mechanical engineering. Working with robots helped me find a confluence for all those interests, and I was fortunate to have the opportunity to work on robot soccer at UT Austin for my master's and Ph.D. which allowed me to pursue other robotics projects such as autonomous vehicles, reinforcement learning, and multi-robot systems.

Even to this day, I like solving many problems and assisting my team in whatever capacity they need. For this reason, I’ve worked on many teams in Sony AI -- including core research, research engineering, cloud engineering, and internal operations. I specialize in introspecting large systems and problems and quickly determining the steps necessary to improve them. Currently, I help lead our cloud engineering effort as well as internal operations at Sony AI.

What made you excited about joining Sony AI and what do you hope to accomplish?
I'm grateful to be part of a small global team of highly talented individuals working on challenging problems. Our team includes people from many different cultural backgrounds who all collaborate well despite barriers arising from being not physically located together and working in different time zones. The team at Sony AI was and continues to be the biggest draw that Sony AI has for me.

My hope is to continue supporting my team as we scale up as an organization, so that they can continue pushing the frontier of AI research while working on domains that further Sony AI's goal of pursuing creativity in AI. I'm incredibly excited about working on new challenges as they arise over the next few years.

How do you express creativity? And what are your thoughts about AI for creativity?
This is a hard question! I'm not entirely sure. I think I get a lot of joy from any work that requires problem solving, precision, and hard work. In my spare time, I work on woodworking projects that help me achieve those goals. While I often express my own vision by working on small furniture pieces, I also often help my wife express her creativity by helping her frame and mount her projects.

As a research community, we haven't quite found the AI applications that truly enrich human lives in a meaningful way. For AI to be truly impactful, we need to work on domains that help it enhance human creativity for everyone. I look forward to the accomplishments Sony AI will have in this space over the next few years.

Piyush completed his Ph.D. in computer science at the University of Texas at Austin in May 2017. After his Ph.D., he started working at Cogitai Inc., which became a part of the Sony AI organization in 2019. Piyush received his undergraduate degree at the Indian Institute of Technology Roorkee in India in computer science and engineering in 2009 and his master's degree in computer science at UT Austin in 2011.

Samuel Barrett

Senior Research Scientist

Based In

Cambridge, MA

Areas of Expertise

Reinforcement Learning

What made you interested in working in AI or your area of expertise?
Artificial intelligence has fascinated me from a young age. I've always liked to imagine the exciting new worlds that AI can enable. In particular, I read a lot of science fiction books as a kid and liked the idea of intelligent robots and computers helping people. I believe that AI has the potential to transform the world and society for the better.

I find reinforcement learning (RL) especially exciting because it allows the creation of agents that you interact with. While a lot of artificial intelligence focuses on learning from fixed datasets, RL enables agents to learn from interacting with the world. In particular, RL can let agents learn complex and surprising behaviors from these interactions. I believe that RL is an important tool for creating agents that interact with the world like people do. We're not close to that happening yet, but RL feels like a useful tool for approaching the problem.

What made you excited about joining Sony AI and what do you hope to accomplish?
Joining Sony AI is exciting from several perspectives. For one, it has a great team of people to work with. I've learned a lot working with my colleagues and it's a great environment to grow and accomplish exciting things.

Also, I believe that games are a very useful tool for learning how to create intelligent agents. Games are easier and more well defined than many real-world tasks, but retain many of the complexities and ambiguities that are important problems for AI to tackle. Another advantage of games is that the cost of failure and experimentation is low, which lets us try out new ideas faster and try out more ambitious ideas.

My dream is to get interesting learning agents out into the world, so I'm particularly excited about Sony's partnerships with gaming companies and expertise in robotics, which will help translate the research we do into real products. In particular, I think that there are a lot of areas in which AI can improve games and increase the complexity and creativity of the players' worlds, allowing them to have complex opponents, teammates, and environments that react to what they do. I'm frustrated by the brittleness of existing AIs in games, so I'd like to have agents that learn and adapt with you.

What is your favorite form of creative expression?
I enjoy the creativity of programming, research, and problem solving. In these areas, you're not solving the same problem every time. Each problem is new and you have to think of new ways to tackle them. That's one of the things that keeps me coming back; the constant need to think of new ideas and new approaches keeps me engaged, learning, and growing. When I take time off, at some point I find myself thinking about the next problems I should tackle and how I should start new projects.

I also really enjoy the creativity of music. I've played flute and oboe for a long time, and I have also written some music. I enjoy listening to music and appreciate the creativity of it. I think that even as listeners, we're engaging with music and each experiencing it in our own ways. It sparks creativity, and I find myself finding new parts of the music that I didn't hear before, or creating my own counterpoint lines. I'm excited about the application of AI to music. I think that it will allow more people to make music and explore their creativity in new ways.

Samuel joined Sony AI in 2019 as a senior research scientist, when Sony AI acquired Cogitai. He previously worked at Amazon Robotics using AI to improve the performance of the robotics system inside Amazon's warehouses. He received his Ph.D. thesis at the University of Texas at Austin in October 2014. His dissertation examined the problem of ad hoc teamwork and cooperating with unknown teammates by exploiting previously learned knowledge. In addition, he helped the UT Austin Villa team win the 2012 international RoboCup competition in the standard platform league (SPL). Before joining UT, Sam received his B.S. in computer science from Stevens Institute of Technology in May 2008.