ALL ENGINE VIRTUAL PRODUCTION
Blog #4
December 3rd 2020
By Gabriella Krousaniotakis
This blog briefly covers these past two weeks in my journey with Virtual Production.
I finished watching the Unreal Engine Virtual Production Primer Core Principles II (second time watching both I & II.)
Continued learning Autodesk Motionbuilder by retaking the amazing course I found for beginners.
In Unreal Engine, I learned how to easily attach a weapon or prop to some new characters I snagged on Marketplace during their black Friday sales.
I did a lot of Unreal Engine Live Link testing with iClone/Live Face, Xsens MVN Animate, and Manus Core.
I performed my first mocap session with the Xsens Link suit and what I can say is that I am now madly in love with doing motion capture live inside of Unreal!
I created a little sequence using everything I learned from the Mograph Winbush Unreal Engine Course with the Xsens mocap data.
Lastly, I met my Creative Pinellas Mentor, Victorial Jorgenson.
UNREAL ENGINE VIRTUAL PRODUCTION PRIMER
I watched the Summer Seminars Core Principles Part I & 11 of the Virtual Production Primer (twice) and have to say, I am so happy that I did! Matt Workman did an amazing job at going over the different aspects of Virtual Production. He broke it down into three categories (CG All Engine / Mixed Reality / LED Volume.)
In addition, he also went over the equipment and software that each one entails. When I first started learning about what Virtual Production was, it was hard for me to wrap my head around it.
After listening to Matt Workman go over everything in detail while showing examples of each, this gave me a clear picture of all the different aspects that make up this amazing new field of filmmaking. Thank you Epic for creating this course! And thank you Matt Workman.
What this does for me, is I now know what part of Virtual Production I am doing. All Engine…and I must say, I love it!!! One thing that Matt talked about that really stood out is how he now has a team from all over the world that he collaborates with. Because there are so many things I am learning, from 3D animation, motion capture, building maps, composure and the list goes on, it makes sense to have people who are skilled and experienced in each of these fields as this type of filmmaking requires a combination of all these skills.
His advice on putting yourself out there, showing what you are doing and sharing it, that is the best advice ever!
MANUS PRIME HAND MOCAP AND UNREAL
Using the Manus Prime II gloves inside of Manus Core software, I was able to test out the Unreal Engine Live Link plugin and animate these hands in real time. Getting the gloves setup in Manus Core software is quite easy. You put the gloves on, hook up the Manus dongle to your desktop and then calibrate the gloves. Installing the Live Link Plugin is also fairly easy thanks to some amazing tutorials Manus has on how to do this.
The team from Manus is truly exceptional. They have been there to help me get everything set up, always there to answer my questions and lastly, seeing actual hand data on a model in Unreal being driven by your own hands in real time…..it is one of the most exciting experiences ever!!!! I never realized how important hand animations are in order to convey emotions. I cannot wait to incorporate the Manus gloves with the Xsens suit this week!
FACIAL MOTION CAPTURE DATA
I tested out the Reallusion iClone Motion Live plugin. I was able to preview my facial expressions on this character in iClone using my iPhone and the Live Face app in the Motion Live plugin. Then I combined sample body mocap data from Xsens MVN Animate streamed live into iClone with live facial data from Live Face. I used the iClone Unreal Engine Live Link plugin to import the body and face mocap data into Unreal Engine and recorded this data inside of Unreal Engine, live from iClone. Being able to do all of this in real time is in my opinion so surreal.
UNREAL ENGINE CHARACTER PROPS & XSENS MVN ANIMATE LIVE LINK TO UNREAL TESTS
I learned how to attach a weapon to a character that I found on Unreal Marketplace. Thanks to a tutorial the MocapGod shared with me and a tutorial made by Hollowswoodsman, I was able to attach a prop to the characters hand. Using Xsens MVN Animate sample data, Live Link to Unreal Engine, I used take recorder to record those animations to this new character and then created these little sequences in the new maps I got on Marketplace.
For the Xsens mocap data, I used one of Xsens amazing tutorials to retarget the animations to these new characters in Unreal Engine and then used Take Recorder to record those animations on each of those rigs…..in real time!
MY FIRST DAY WITH THE XSENS LINK SUIT
My first day using the Xsens suit was on November 30th! When the case from Xsens arrived, I knew my life was about to change. Inside this case is what dreams are made of. This company has thought of everything! Even down to the measuring tape and extra lycra suit (how Katie Jo knew the Large was going to fit better than the Medium I thought I was going to fit into still amazes me!)
I did my body measurements, set up the suit and sensors (thanks to those amazing tutorials they have), and the best part was Katie Jo from Xsens did a video chat with me and walked me through the entire process on how to get everything to work from setting up the ASUS, how to turn the battery on the suit on and off, how to set up my body dimensions, how to calibrate, and how to record data. Since that meeting, I have been using the suit every single day and having a blast!
I think what makes this experience so exhilarating isn’t just this unbelievable technology, but it is the interaction with Katie Jo. When I first contacted Xsens, Audrey Stevens sent over some past seminars for me to look at, which I did, in awe. Seeing Katie Jo talk about the product and motion capture, that enthusiasm, that excitement, that is what has stayed with me. When someone loves what they do, I mean they truly love it, you can tell. And you can also recognize that in others if you yourself love what you are doing. Finding someone that shares that excitement is where the joy comes from because you have someone you can share that with. I get emotional just thinking about it.
In the last 72 hours of having the suit, my entire life has changed. I have changed. My level of creativity and confidence in executing what I am seeing in my mind and translating that into Unreal Engine, it is UNREAL!!!! I created a sequence that even I am shocked I produced. I mean, this equipment and support has literally made my dreams come true. Amazing. If this is a dream, I never want to wake up. I can honestly say this is the happiest I have ever been in my entire life. The only thing that tops it is adopting my little dog Enzo!
Quality content that can be created in Unreal using an Xsens suit, the MVN Animate software (that blows my mind at how well made it is) along with the outstanding support from the people of Xsens is INFINITE!
Once I had the suit calibrated, I used the Xsens MVN Animate software and did a Live Link to Unreal Engine so I could see what my body movements looked like in Unreal, already attached to a rig.
I found this especially helpful because I learned about my spacing when wearing the Link suit. If I brought my hands too close to my body, I noticed later that my hands were going into the character mesh, so the ability to test out the Xsens mocap animations I was doing in real time on a rig inside of Unreal, I was able to redo my recordings while watching what those looked like on my Unreal Character. The first video shows my tests. The second one shows the sequence I created after rerecording my animations and adjusting my hand spacing.
MOTIONBUILDER LEARNING
Some clips from the Motionbuilder course I took in order to learn the basics. I used sample BVH files from Carnegie Mellon University and for the Gangnam Style mocap, I used the Xsens sample fbx file. Lastly, I imported an Unreal Engine character into Mobu, characterize her and then added a prop to her hand and a BVH animation. My first attempt at this.
CREATIVE PINELLAS MENTOR
I had the absolute pleasure of meeting and talking to my new mentor, Victoria Jorgenson. Victoria is a Producter, Director and Writer based in Pinellas County and has been in the Film Industry for over 20 years. Her story was fascinating to me and so is her work. That may be because I am finally interacting with people that are on a path I have spent my entire life dreaming about and now I actually get to talk with them!
I have to thank Creative Pinellas for pairing me up with Victoria. I have learned so much from her already and she is pulling out all the stops in order to help me. Moving mountains, more like it! Now where do you find people that go out of their way to help you bring your visualization to life? Apparently, Pinellas County!
Victoria, if you are reading this, know that you have a friend for life. I will never forget what you have done for me already and in such a short amount of time. I never expected this mentorship program to be so incredibly exciting. You are opening up all these doors for me, doors I never knew existed. And your generosity has me humbled.
