On a recent afternoon in Tokyo, Sony AI Director and Lead Engineer, Peter Dürr, alongside project team members, watched a table-tennis ball vanish and reappear in front of him with narrowed eyes.
On the one side of the net, professional Japanese table tennis player, Taira Mayuka, launched a smash that would normally decide the point. The ball cleared the net in a blur, dipped hard toward the corner and, somehow, came back.
On the other side of the net, Sony AI’s Ace robot had read the trajectory, adjusted the racket angle and answered with a return that kept the rally alive.
Dürr clenched his fists, “Yes!,” he exclaimed.
Caption: (top left), Sony AI Director and Lead Engineer, Peter Dürr, celebrates an Ace victory. (top right) Mayuka Taira prepares for Ace’s serve. (bottom) All Project Ace matches adhere to the International Table Tennis Federation (ITTF) rules. These regulations cover every aspect of play, including the dimensions of the table, ball specifications, and service requirements. Shown here is an official referee monitoring a match in Tokyo.
“We wanted to prove that AI doesn’t just exist in virtual spaces,” said Michael Spranger, President of Sony AI. “It’s not just tech you interact with in the virtual world—you can actually have a physical experience, and the technology is ready for that.”
Caption: In April 2025, Ace competed against five elite players and two professional players in the milestone evaluation underlying our Nature publication. All matches followed International Table Tennis Federation (ITTF) rules and were refereed by licensed umpires from the Japan Table Tennis Association ensuring regulation scoring and independent oversight.
For more than forty years, roboticists have chased a classic challenge: how to build a machine that can rally with a human. And not only that, one capable of perceiving, reacting to, and returning the blistering speed and spin of elite-level table tennis.
Since the first “robot ping-pong” prototypes appeared in 1983, hundreds of designs have tried to match the reflexes, precision, and adaptability of top athletes including speed.
“Speed in robotics that is not predetermined is one of the last frontiers in robotics,” Spranger said. “Robots are still slow. It’s difficult for them to interact with the environment—let alone with people—at the speed we are accustomed to.”
No robot has ever been able to see, decide, and act within that razor-thin window—until now.
Sony AI’s Ace meets this challenge.
Published on the cover of Nature, this research introduces the first real-world AI system capable of competing with and beating elite university and professional table tennis players under official rules. Indispensable to the project were the elite and professional table tennis players, coaches, past Olympians and industry experts whose expertise informed system evaluation and match design.
Through a combination of high-speed event-based vision, low-latency control, and reinforcement learning, Ace tracks and returns high-spin, high-velocity shots in real time. With an end-to-end latency of 20.2 milliseconds—compared to roughly 230 milliseconds for elite human players.
Over five years, Ace progressed through a series of milestones, from early experiments simply keeping the ball in play to competing against high-level players.
“It started with juggling the ball,” Spranger said. “Then, cooperative rallies where a human works with the robot to keep the rally going. From there we moved to playing against increasingly stronger table tennis players. Seeing that progression—the technology improving, the team coming together, celebrating those milestones—has been incredibly rewarding.”
Behind those achievements lies a series of breakthroughs in physical AI.
Ace’s architecture integrates nine synchronized frame-based cameras and three event-based vision systems—produced by Sony Semiconductor Solutions—to track a ball at 200 Hz with millimeter accuracy and around 10 ms latency while measuring the spin at up to 700 Hz. This is fast enough to capture motion that would be a blur to the human eye.
Its deep reinforcement learning control (trained entirely in simulation) transfers seamlessly to real-world play, producing shot variations and adaptive rally behavior. A custom-built eight-degree-of-freedom robotic arm, manufactured using optimized lightweight alloys, delivers human-level reach and acceleration while remaining safe and precise.
Together, these advances close a gap that has persisted since 1983, marking a milestone for real-time human–robot interaction and high-speed physical intelligence.
Ace began as one of the earliest research efforts at Sony AI.
“The beginnings of the project were really humble,” Spranger said. “There was nobody. There was no office. It was one of the first projects we started at Sony AI in 2020. The first person I recruited for this project was Peter,” he added. “I thought he was the person who could really make this successful.”
Ace also sits within a longer tradition of milestone demonstrations in artificial intelligence. In 1998, Deep Blue defeated Kasparov. In 2016, AlphaGo mastered intuition. In 2022, Sony AI’s GT Sophy reached super-human control in Gran Turismo.
Ace continues that lineage, marking the first time an AI system has achieved human expert-level play in a commonly played competitive physical sport. Ace also takes that progression from digital into physical spaces.
As Peter Stone, Sony AI’s Chief Scientist explains, “This is bigger than table tennis. It takes its place in a series of AI landmarks. It’s the very first time there’s been a human expert-level demonstration of competitive play in the real world across any sport—not just table tennis.”
Ace extends Sony AI's reinforcement learning expertise from simulation to the physical world. But both GT Sophy and Ace share the same learning philosophy: develop agents that perceive, decide, and act autonomously under uncertainty.
"We built a learning mechanism where the AI is trained in simulation and then transfers to the real world," Spranger said. "The robot plays against itself to improve, and then we bring that learning into reality."
GT Sophy mastered precision control within simulation; Ace brings that intelligence into the real world, where milliseconds, sensing and physics matter.
Ace draws from GT Sophy in one unexpected way. During training, Ace's learning system used a technique called a privileged critic, which is a component that, in simulation, has access to perfect information that would never be available in a real match. The policy itself learns only from realistic sensor inputs, as it would on a real court. But with a better-informed critic guiding it during practice, the policy gradually learns to fuse its own sensor data and anticipate the ball's trajectory, without being told how.
Dürr had seen a version of this approach work in GT Sophy, but didn't expect it to translate to a physical system.
"It totally blew my mind," he says. "I didn't think this was possible at all—but with this kind of privileged information fed to the critic, it turns out the policy can learn how to do sensor fusion and anticipate the trajectory of a table-tennis ball."
GT Sophy proved that reinforcement learning could master expert-level control in a complex, high-speed environment. Ace proves it can also do so in the physical world, where sensing, latency, and hardware add a new layer of challenges.
Ace’s learning architecture unfolds across three layers:
Skill >> How to move joints and generate spin or power in real time.
Tactics >> How to respond within a rally, deciding placement and pace.
Strategy >> How to plan over the course of a match.
“Our emphasis for this Nature paper is on the skill, and that’s where most of the reinforcement learning is. There’s still room for improvement at the tactics and strategy level,” Stone explains.
He also notes that these layers reflect different horizons of decision-making:
Skill >> governs immediate motion.
Tactics >> govern point-by-point choices.
Strategy >> governs how play evolves across an entire match.
Dürr reinforces this distinction from the engineering side.
For him, rally performance begins with accurate, low-latency state estimation, meaning, knowing where the ball is, how it spins, and how fast it travels.
This information feeds directly into the learned skill policies that control each joint of the robot.
“Basically, what we need to do is estimate the ball’s position and spin and then the policy can learn how to do this kind of task,” he says.
Where the two perspectives converge is Ace’s ability to adapt while maintaining stability.
To reach elite-level performance, Ace blends these layers seamlessly.
Skill ensures the ball is struck cleanly. Tactics determine whether it is struck safely or aggressively. And strategy determines how these decisions accumulate over a match.
From the beginning, researchers built-in a safety process into the project.
“Initially, people were wearing helmets and pads and glasses. And as the robot became more and more validated, and we had safety certification come through, we were able to remove some of those things. And that was a really important piece of making this project successful. For the first time at this speed, robots are actually interacting with humans” Spranger explained.
Caption: In the early days of the Ace Project, players competed with safety gear until Ace was validated as safe for human play.
The next challenge was perception: sensing the ball quickly and accurately enough to react in time. Ace therefore combines conventional frame-based cameras with event-based vision sensors. If frame cameras are like photographs taken in rapid succession, event-based sensors are more like motion detectors that fire only when something changes.
In table tennis, spin is often the deciding factor in a rally. It determines how the ball curves through the air and how it rebounds off the table, making accurate spin estimation critical for predicting the ball’s trajectory.
Ace’s hybrid sensing approach allows the system to track the ball’s position, velocity, and spin at very high temporal resolution.
As the researchers note, accurate state estimation under these conditions is essential, because “errors in ball state estimation quickly lead to missed or unstable returns.”
In practical terms, the system must know not just where the ball is, but how it is rotating and how that rotation will alter its flight after the bounce.
Ace’s striking skills are trained entirely in simulation using reinforcement learning, then transferred directly to the real robot. This is analogous to a player who practices endlessly in a virtual training hall and then walks onto a real court without needing to relearn anything, not just the basics. According to the paper, the learned policy transfers “without additional fine-tuning on the real system,” despite the extreme speeds involved.
All of this learning would fail, however, without hardware that behaves predictably. The robotic arm used by Ace is custom-built, not off-the-shelf, and designed specifically for high-speed, repeated impacts. Its mechanical properties—stiffness, low vibration, and precise actuation—ensure that when the policy issues a command, the robot responds consistently.
The research stresses that “accurate modeling of the robot dynamics” is a prerequisite for successful simulation-to-real transfer.
The team progressively improved our physics models of ball dynamics as the other parts of the system improved. As the perception system became more accurate, it became possible to design physics models based on cleaner data, which enabled the team to uncover complex, previously hidden dynamics more accurately. As the physics models became more accurate, the robot trained on the improved simulator became more competitive, which enabled them to collect more data in higher speed and spin regime and tune the models to cover a wider range of ball states.
Taken together, these advances form a single system rather than a collection of components. Low-latency sensing enables rapid state estimation; rapid estimation enables learned control; learned control depends on predictable hardware. The contribution of Ace lies not in any one of these elements alone, but in demonstrating that, when integrated carefully, they are sufficient to support sustained, expert-level competition in a real-world physical sport.
Every AI milestone depends on timing, when computation, engineering, and imagination align. “Just like Deep Blue had the right compute in 1998, we had the right team, hardware, and innovations at this unique moment in time,” Stone explains.
And his point is simple: these breakthroughs are not inevitable; they are fragile intersections of capabilities and opportunities.
Ace represents years of convergence across physics, perception, control, sensing, and reinforcement learning—a convergence Dürr also emphasizes. In his view, success depended on understanding both the physics of table tennis and how a robot moves when we send a command to it.
These elements (accurate sensing and physic modeling, and predictable actuation) created the foundation that allowed reinforcement learning to transfer from simulation into real-world play.
This is also where Ace diverges from past attempts, including earlier robotics projects and the well-known work at DeepMind. Dürr explains that earlier systems often relied on high-quality cameras and off-the-shelf robots but did not deeply optimize perception or robot hardware.
Sony AI, by contrast, invested across real-world simulation, sensing, hardware, and learning. The system combines custom event-based sensors from Sony Semiconductor Solutions, a purpose-built robotic arm, and a reinforcement learning architecture designed for simulation-to-real transfer.
Ace was born of shared ambition across a global Sony AI research team.
The road was full of moments when success was uncertain, and the team had to push through periods where it wasn’t clear this was going to be possible.
Dürr’s perspective mirrors this. Though much of the public attention centers on visible breakthroughs, like returns, smashes, precision shots, Dürr underscores that years of painstaking iteration on perception, hardware tuning, and physics modeling made those moments possible.
For example, the team only recently discovered that their physics model overestimated the aerodynamic drag on very fast shots, an insight that emerged only after playing increasingly strong human opponents.
“We started playing harder and harder smashes and we discovered the model predicts the ball will descend quicker than it actually does,” Dürr explains.
That process of discovery extends beyond physics.
Guilherme Jorge Maeda, Research Scientist at Sony AI (Tokyo) and a co-author on the Nature paper, leads robot serve design and system performance evaluation. He notes that evaluating Ace is inherently non-deterministic because the opponents are human.
Caption: The featured Project Ace Team: Abecassis, Lison, AI Engineer; Adodin, Pavel; Aydin, Asude, Computer Vision Engineer; Bi, Yin, Senior Research Scientist; Blakeman, Sam, Staff Research Scientist; Conti-Fujiwara, Christian, Senior Research Scientist; Farshad, Khadivar, Senior Research Scientist; Fuentes, Dunai, Senior AI Engineer; Giammarino, Alberto, Robotics Engineer; Grover, Divij, Robotics Engineer; Heusser, Stefan, Staff Mechanical Engineer; Hu, Yunpu, Senior Research Scientist; Huang, Yuting, AI Engineer; Kreiser, Raphaela, Senior Research Scientist; Maeda, Guilherme Jorge, Staff Research Scientist; Monferrato, Valentin, Research Assistant; Mukai, Nobuhiko, Staff Engineer; Nagel, Yannik, Robotics Engineer; Sahloul, Hamdi, Computer Vision Engineer; Saraiji, Yamen, Staff Research Scientist; Schilling, Fabian, Senior Research Scientist; Scotti, Andrea, Software Engineer; Silva, Tiago, Software Engineer; Takahashi, Naoya, Senior Staff Research Scientist; Tapiador-Morales, Ricardo, Senior Research Scientist; Torrente, Guillem, Senior Robotics Engineer; Walther, Etienne, Robotics Engineer; Yang, Bilan, Jackie, AI Engineer; Ynocente, Mario, Senior AI Engineer.
“Every player plays in a different style,” Maeda explains, “and we always learn something new from a different opponent—or they reveal flaws that weren’t visible before.”
He adds that even day-to-day factors, like warm-up, fatigue, or confidence, can meaningfully affect outcomes, making performance evaluation a moving target rather than a fixed benchmark.
Setbacks, he says, are part of the process. “I have never considered losing games as a real setback or failure,” Maeda notes, “but just as an intrinsic part of playing a competitive game.”
These discoveries shaped each successive design decision. The team’s culture of experimentation and comfort with being wrong before being right proved essential.
Researchers across the perception and control teams developed the privileged-critic reinforcement learning architecture that ultimately convinced the group to adopt the approach.
“I didn’t believe in the privileged-critic approach at the beginning,” Dürr recalls. “And then I saw the results,” Dürr recalls.
Those internal experiments accelerated Ace’s evolution.
This cross-disciplinary ethos, (combining learning algorithms with physical engineering), has become a defining aspect of Sony AI’s approach for this research. Together, these contributions shaped Ace into a system capable of competing with elite human players under real-world conditions.
Reaching expert-level play marks a milestone, not an endpoint for the future research in this domain.
Stone captures this clearly: “This isn’t perfection yet. There’s still room for improvement in the hardware and in the strategy.” He emphasizes that Ace has reached elite-level performance, but not yet the level of world champions. “Some people are still better than this system,” he says. A distinction the team is up-front about.
Stone explains that the version described in the paper was not yet hitting with the same force or spin that elite humans generate: “It was still not quite hitting the ball as hard or with as much spin as people… but now it is,” he notes, referencing improvements made between April 2025 and December 2025.
Dürr sees similar headroom.
For instance, the system still tends to hit earlier after the bounce than human players, even for shots that would benefit from moving back from the table, a behavior that “just emerged” from the learning process and is “something that we don’t particularly like.” He notes that this early-hitting behavior may narrow Ace’s shot variety, and the team expects the system to learn more human-like timing as exploration improves.
Dürr’s account of simulation-to-reality transfer points toward the same frontier.
Even with exceptionally accurate sensing and a carefully tuned physics model, certain real-world shots (especially extreme smashes) still reveal subtle mismatches.
“In simulation, we hit the ball on the table. In reality, it flies too far,” he explains, citing ongoing work to reconcile those discrepancies.
From Maeda’s perspective, this gap highlights why Ace represents a new form of physical intelligence. While AI has long surpassed humans in symbolic or cognitive domains, he notes that Ace demonstrates competitive performance in a physical intelligence setting—where uncertainty, timing, embodiment, and interaction dominate.
Ace’s impact may also lie in the advancements in each of its component technologies: faster perception systems, low-latency control loops, and new simulation-to-reality transfer methods.
Stone compares Ace to historical technological moonshots: “Like the Apollo mission, it’s not about immediate commercialization but the technologies that come out of it.”
He notes that major scientific leaps rarely become products overnight. Instead, they accelerate entire fields.
Stone points out that many of Ace’s breakthroughs are most valuable because they solve general AI-and-robotics problems: “There’s nobody else who has such a robot… none of the past attempts had a robot that was as good, or tracking that was as good.”
These capabilities could readily transfer into new domains.
Even Dürr’s engineering challenges–like reconciling simulation with real-world drag, or designing vibration-resistant lightweight structures—represent reusable insights for next-generation autonomous robots.
Taken all together, Ace’s contributions form a technical foundation for broader progress in physical AI. These contributions could also have potential applications in assistive robotics, rehabilitation, adaptive training, and other human-centric fields.
Beneath every technical milestone in AI research lies a deeper question: What is the nature of intelligence?
“And what is the nature of intelligence?” Stone asks. He argues that physical AI represents a frontier where this question becomes experimentally testable.
Until now, he notes, AI accomplishments largely clustered around symbolic tasks (chess), language or pattern tasks (Jeopardy!, Go), or virtual control tasks (GT Sophy). Physical sports had not yet attained this level of play.
Ace demonstrates that intelligence can be studied not just through thought and simulation, but through physical interaction, where sensing, motion, decision-making, and prediction must converge under strict time pressure.
Ace brings together robotics, machine learning, and physics in a system designed to operate at the speed and complexity of real-world sport.It is a system that sees faster than humans, reacts faster than humans, and competes against the very athletes whose skills inspired its creation.
Caption: Each opponent is afforded an appreciation award from the Sony AI Project Ace Team to celebrate their competition against Ace.
“At Sony AI, we’ve always focused on exploring research projects from two angles,” Spranger said. “One is we want to push the technology on a technology barrier. We want to overcome what’s possible today and provide the next step for the technology, but we want to do that together with artists, engineers, researchers, the sports industry, in order to augment their behavior or their capability rather than replace them.”
As Stone notes, Ace is not the end of robotics in sport; it is the beginning.
As the team’s experiments progressed, many of Ace’s advances came not from single breakthroughs but from years of incremental improvements in sensing, control, and learning. The work also builds on decades of research in robotics, control theory, and machine learning, as well as the expertise of collaborators on this project such as Victas and the players who competed against Ace.
Ace is a research milestone. It opens new scientific questions, new engineering pathways, and new possibilities for how humans and intelligent machines may interact in high-speed, high-stakes environments.
The frontier remains incomplete which is precisely what makes it exciting.
And we encourage you to read the paper in Nature and watch our short film exploring Ace in the lab. To learn more about Ace and read the research, visit ace.ai.sony
To explore the research further, visit: