Keynote 2

Title: Training the World’s Best Gran Turismo Racer

Speaker: Dr. Pete Wurman, Director, Sony AI, USA

Abstract: Automobile racing represents an extreme example of real-time decision-making in complex physical environments. Drivers must execute complex tactical maneuvers to pass or block opponents while operating their vehicles at their traction limits. Modern racing simulations, such as the PlayStation game Gran Turismo, faithfully reproduce much of the nonlinear control challenges of real race cars while also encapsulating the complex multi-agent interactions. In this talk, I will describe how our team at Sony AI trained agents for Gran Turismo that can compete with the world’s best e-sports drivers. We combine state-of-the-art model-free deep reinforcement learning algorithms with mixed scenario training to learn an integrated control policy that combines exceptional speed with impressive tactics. In addition, we construct a reward function that enables the agent to be competitive while adhering to racing’s important, but under-specified, sportsmanship rules. We demonstrate the capabilities of our agent, Gran Turismo Sophy, by winning a head-to-head competition against four of the world’s best Gran Turismo drivers.

Bio: Pete Wurman is the director of Sony AI America and project lead for the Gran Turismo Sophy project. He has done research in computational auctions, multi-agent systems, robotics, and reinforcement learning. He earned his Ph.D. at the University of Michigan and was a professor at North Carolina State University. He was the Co-founder and CTO of Kiva Systems, the company that pioneered the use of mobile robotics in warehouse fulfillment operations. Kiva was acquired by Amazon in 2012 and, as Amazon Robotics, has deployed more than 500,000 robots to warehouses around the world. While at Amazon, Pete founded the Amazon Picking Challenge. He is an IEEE Fellow and a AAAI Fellow and was inducted into the National Inventors Hall of Fame in 2022.