New AI: This Is A Gaming Revolution!
❤️ Check out Lambda here and sign up for their GPU Cloud:
📝 The paper "Categorical Codebook Matching for Embodied Character Controllers" is available here:
📝 My paper on simulations that look almost like reality is available for free here:
Or this is the orig. Nature Physics link with clickable citations:
🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:
Alex Balfanz, Alex Haro, B Shang, Benji Rabhan, Gaston Ingaramo, Gordon Child, John Le, Kyle Davis, Loyal Alchemist, Lukas Biewald, Martin, Michael Albrecht, Michael Tedder, Owen Skarpness, Richard Sundvall, Taras Bobrovytsky, Ted Johnson, Thomas Krcmar, Tybie Fitzhugh, Ueli Gallizzi.
If you wish to appear here or pick up other perks, click here:
My research:
X/Twitter:
Thumbnail design: Felícia Zsolnai-Fehér –
What a time to be embodied 😅
*Aint that a Kinect? 😅*
Yes! A kinect without the kinect
This way you don’t need anything to be recording you, just the headset and controllers. No console, no accessories. You could take a walk with this outside and it would still work. You could bring it to a friend’s house, put it on and it’ll still work.
completely different methods
@@therealpeter2267 Except that you need to feed it with 3 hours of training data i.e. recordings of your motions.
@@CIinboxStill doesn’t record you as in video recording.
add this with some extra knee or feet sensors, we’ll get the perfect embodiment tracker!
i wonder what the cost difference is (between ml generation, to the more analogue solution of like extra sensors for knees/ankles)
@@c4blecdepends on what kind of tracking solution you use. The lowest would be about 300 euros with something like slimeVR trackers.
The metaquest 3 can get a decent view of the legs at least when you’re not standing straight up.
Certainly what I would consider good enough.
@@c4blec What do you mean by “cost”? Money or computation effort? Either way, good question. Right now I use 11 point tracking and it was not cheap to buy. It keeps up with my movement 99% of the time though. I use foot, knees and hip for lower body. You want the tracker on the foot not ankle, so it captures foot twist for movement like the C walk or shuffle or whatever dance move you are doing. This ML/AI is a long way off from full capture.
@@c4blec at this point its not about the cost but accuracy, there is only so much you can do with guessing, like no way this could handle someone doing yoga in VR without extra data about legs – and probably would break much quicker with just unusal doge positions or kicks.
But its still amazing solution to just make games feel more responsive/natural without adding much cost and burden, for games where legs are not important but you still want them to be visible.
what a time to be alive!
So they used neural networks to improve the accuracy of inverse kinematics for VR?
Yea this seems like a c- paper. Cool but not groudbreaking
*and predictions of future movements
Seems like that’s what they did. It’s very impressive how well it worked though. It’s probably not easy to get this level of accuracy.
Sounds more like a system made from the ground up rather than adapted from IK. There is probably still endpoint control to plan the animations, but it doesn’t look at bone length at all when planning the motion it seems. The motions aren’t rigidly slaved like with IK.
@@JustAPersonalUseBarb interpolation forward in time isn’t new. I know hacks for videogames that do this. Admittedly as the other guy says this ISNT inverse kinematics but it DEFINITELY follows the same principle.
This would probably really benefit from one more sensor to clip to your belt. Acceleration, hip angle, vibration from steps would make lower body guesses much more accurate without major inconvenience.
Yup! That’s how Standable on Steam works. It just fills in the blanks of whatever trackers you don’t have.
Ring Fitt for the Nintendo Switch did something similar. One of the controllers slid into a little Velcro strap that wrapped around the player’s leg.
I’m genuinely impressed with it’s leg motion simulation and how smooth it is. Exciting updates as always. Thank you as always.
It is pretty incredible, this will be very helpful in the near future
Which company is devoloping this tech? thanks!
There was that time when Kinect, PS Eye and Wii were so popular, but I can only guess that the money dried up when the novelty did, and then they all just died slowly.
The one thing I really enjoyed was the fitness games where the camera tried to match your body to the correct posture. Also, I’m guessing family/party games don’t really sell. Game devs would have to use this technology in FPS games or some popular category and find a way to make this a core mechanic of the game without it being gimmicky or wearing out the novelty.
I’m still using the Kinect to play sports games in the winter when I don’t want to go outside. Some Highscores and pretty pictures are good workout motivation.
A used Kinect is also quite affordable nowadays. The online store will be shut down soon though.
Try VR. Boxing, tennis, golf, mini golf, bow and arrow, and Beatsaber.
1:33 “That is absolutely insanity.”
I concur.
It’s not. This is basically just slightly better inverse kinematics, nothing revolutionary here.
We finally have the leg technology for the metaverse, surely this will save it
I wonder if it’s possible to use AI to generate the view of a slightly different perspective, so VR games would not need to render an image for both eyes.
Hm, sounds like standard POV to stereoscopic images could be done. But we would be dealing with exactly the same issues we see in DLSS (spatial/temporal stability and disocclusion artifacts)
You don’t need to. There is a feature called “instanced stereo” it basically re-projects one eye’s render onto the depth map of the other.
Those are the legs Zuckerberg desperately needed for Meta back then
Right now any VRchat fan without the budget for (or faith in) body trackers should be very excited. Plus some of my favorite games like Into The Radius and Blade & Sorcery would seriously benefit from this.
they wouldnt cause you cant kick with this
04:36 I came for the information. I smiled for the moment.
This will be great for online VR gaming imersion, where you can see the other players doing more realistic movements,
for example if your playing a shooter and the enemy is crouching, it should look like a more realistic crouch and therefore more immersive for you.
Wow, adding this to VRChat would do wonders. Just like networks physics collision with other players brings a whole other level of emersion, i think this would be one of those things that one cannot look back from.
I was thinking the same
The video title and thumbnail are appallingly bad for what this video is trying to show off. The content is great and I’m sharing it around to other VR devs, but damn if the title and thumbnail way off the mark :
Welcome to the world of “AI News” . Every little thing is revolutionary, stunning, and insane.
i agree
The title is for the algorithm to read, not you. 😀
During the Friends theme song I always wait for the applause, to clap too.
For years now in every video I’ve been waiting for you to say *”What a time to be alive”* so I can say it too.
I don’t know, maybe I have OCD but… I enjoy it too much!
I Just found this channel again after being lost for a year. I don’t know why I wasn’t subbed before, but I am now. I love your videos. What a time to be alive.
I remember doing this like a decade ago with my Kinect 360…