An artificial intelligence model developed by a couple of PhD students from Stanford created a video game after ‘watching’ tennis matches on video.
Here’s how it works: The AI, called Vid2Player, converts broadcast video of tennis matches into ‘sprites’ of actual tennis players like Federer and Djokovic. These sprites are controllable characters generated from the video. The video game characters move and play just like the real tennis player.
Why is this important?
In essence, this is a deep video fake. The usual way to program video games is to create the characters from scratch and program their look and movement. With this technology, an AI system creates video game characters that look and play like their IRL counterparts.
Style and tennis ball physics
And, there’s more. The AI also replicates the way that the real players would play. In other words, the AI copies the style of the player. And this comment came from expert tennis players reviewing the system.
Not to mention that the AI system recreates all of the physics of the motion of the ball, so everything meshes, just like IRL.
What’s the technology?
The AI watches the video and builds a database of annotated clips like this:
But that’s just the start. It then chooses the right clip for the action that is about to happen. And it corrects the video clip for lighting so that everything looks natural. You can read more detail on the tech in the paper.
Key points to remember to make you the smartest one in the room. This is potentially a new way to program realistic video games and also has the potential for deep video fake abuse to watch out for.