Tennis Anyone? This AI Watched Tennis Matches and Then Turned It into a Video Game

September 30, 2020

An artificial intelligence model developed by a couple of PhD students from Stanford created a video game after ‘watching’ tennis matches on video.


Here’s how it works:  The AI, called Vid2Player, converts broadcast video of tennis matches into ‘sprites’ of actual tennis players like Federer and Djokovic.  These sprites are controllable characters generated from the video.   The video game characters move and play just like the real tennis player.


Why is this important?
In essence, this is a deep video fake.  The usual way to program video games is to create the characters from scratch and program their look and movement.  With this technology, an AI system creates video game characters that look and play like their IRL counterparts.


Style and tennis ball physics
And, there’s more.  The AI also replicates the way that the real players would play.  In other words, the AI copies the style of the player.  And this comment came from expert tennis players reviewing the system.


Not to mention that the AI system recreates all of the physics of the motion of the ball, so everything meshes, just like IRL.


What’s the technology?
The AI watches the video and builds a database of annotated clips like this:

But that’s just the start.  It then chooses the right clip for the action that is about to happen.  And it corrects the video clip for lighting so that everything looks natural.  You can read more detail on the tech in the paper.


Key points to remember to make you the smartest one in the room.  This is potentially a new way to program realistic video games and also has the potential for deep video fake abuse to watch out for.

Easy peasy to share this story with your peeps

Level up your inbox with The Scroll

Get stories like this delivered to your inbox.

Business news focused on startups and tech. Get informed while being very mildly entertained.
No spam. No fluff. No nonsense. Ever.