Working with Face, Body and Hand Motion Capture Data

Gabriella Krousaniotakis / December 11th, 2020 Blog#5

This week I tested out Faceware Studio software in Unreal Engine. Learned about the virtual camera tool and Live Client plugin Glassbox offers. And I also combined the Xsens Link body suit with the Manus Prime II gloves for the first time. Using the body and hand data, I rendered out a short cinematic.

Faceware Studio

I have to start out by thanking that Mocap God for telling me about Faceware and Katie Jo (who I am beginning to believe is my guardian angel) from Xsens for putting me in touch with Faceware. A special thank you goes to Karen Chan and Josh.  Karen went over what all their different software’s can accomplish (Faceware Studio, Analyzer, Retargeter and Shepherd) while explaining the various pipelines the software’s are used in. Josh walked me through a screen share on how to set up an actual character blueprint, in order to get Faceware Studio to work in Unreal Engine using the Glassbox Live Client plugin.

If that is not outstanding service, I don’t know what is. I don’t even think the word “service” can describe these interactions.

Ever since I started to learn about motion capture, I realized what a small niche it is, and that every single person I have met in this community, I find absolutely fascinating. They all share one thing in common, they all get so excited when they talk about motion capture and animating. After spending one week with the Xsens Link suit, I get it now. I cannot express how exhilarating it is to be able to create with motion capture. The creative possibilities are unlimited. You imagine something in your mind, and then create it using motion capture data and Unreal Engine.

What an amazing world I have stumbled into.

I learned that in order to get amazing face data, it really does come down to the face rig you use and the software that translates that data. I realized this in the first 10 seconds of opening up Faceware Studio. After having tested out the other face mocap software’s, apps and plugins, nothing can compare to this one.

Glassbox Dragonfly

I learned about Glassbox through this experience. There are some truly brilliant people out there. Glassbox has a plugin called Live Client that allows you to stream live data from Faceware into Unreal Engine. That, in itself is insanely cool! The other thing they offer is a virtual camera toolkit called Dragonfly for Virtual Production, that you can use in Unreal Engine.

I watched a seminar that showed how three people (Katie Jo, Chris and Norman) were all inside of Unreal Engine from different locations of the world. Norman Wang, from Glassbox, controlled the virtual camera from where he was located and recorded Katie Jo and Chris who were in the Xsens suits and controlling their avatars in Unreal, all of this in real time!

I was speechless. Being able to accomplish all of this, remotely, is a feat in itself. I cannot wait to test out Dragonfly!!

Combining Body and Hand Mocap Data with Unreal Engine

I was already having so much fun using the Xsens Link suit in Unreal Engine, but when I combined the hand data from the Manus Prime II gloves, that was a game changer. After using the Xsens Link and Manus gloves together, I felt complete! My creative process changed. I was not only thinking about what I wanted to perform and record actions and reactions but also how my hand gestures would enhance this.

I  learned through using the Xsens suit with the Live Link to Unreal about body spacing. When using the Live Link to Unreal with MVN Animate, I could test out my body movements and tell whether or not my arms and hands were going into the characters mesh. I could direct myself using this reference when performing the motions I wanted to record and get 100% better results.

 

I am finding the MVN Animate software, the Xsens suit and the Manus gloves are indispensable tools for virtual production and creating in Unreal Engine. I cannot imagine not having them anymore! My entire work process has not only sped up, but the level of creativity and ability to execute what I want, is unbelievable. I am forever in love with these tools.

Creative Pinellas Mentorship Program

I have been truly blessed with my mentor Victoria Jorgenson.

She has gone above and beyond to help me, since I first met her. She has already introduced me to some incredible people! I had the pleasure of meeting Esteban and Meike from Echo Bridge Studio, located in St. Petersburg, Florida. Thank you, Esteban and Meike for your time and your help. Meike was kind enough to put me in touch with the lovely ladies, Alejandra and Jasminka from Ringling College of Art and Design, located in Sarasota, Florida.

Victoria, thank you so much for making all of these introductions to people I would have otherwise never known about. All of them amazing people and part of a community I love so much!