Xsens MVN Analyze/Animate, developed by Xsens, is the main software tool for interfacing with the MVN inertial motion capture suit. MVN Analyze/Animate allows the export of motion capture data to third party applications, making the data available to drive rigged characters in animation and more. The data transfer to other applications is primarily file-based (export) when using MVN Analyze/Animate.
However, in many scenarios, it is attractive to keep the ease of use of MVN Analyze/Animate, but still being able to receive and process the motion capture data in real-time in another application, even on another PC, possibly physically remote from the MVN system.
To this end, MVN Analyze/Animate can act as a server on a network and stream motion capture data in real-time to a client PC running a client application. This document specifically treats the use of a network streamer to stream motion capture data in real-time into the Unreal Engine.
Note: In this workflow, we assume both MVN Analyze/Animate and UE4 are properly installed and the user has a good overall knowledge of working with both programs. Throughout the workflow, Quickoverviews are given in the form of GIFs at the start of the procedure. Pictures are placed AFTER the corresponding text.
1.1 What's improved in this release?
The MVN Unreal Live Link plug-in has been updated to support UE4 version 4.23, post-fix of 14-02-2020. This version included a new way to easily create a retarget asset from your skeleton and the option to package the project with MVN Live Link integration.
2.1 Install plug-in
1. Go to the Epic Store Launcher
2. In the Unreal Marketplace search for MVN Live Link and download the plug-in.
3. In the Library, Go to Vault to install the plugin to the engine version you use for your project
2.2 Enable Plugin
1. Create a new project or open an existing project.
2. Go to Edit > Plugins > Installed > Animation. Here you should see LiveLinkMvnPlugin. Make sure to enable the plugin. This will prompt a restart.
3. GETTING STARTED
3.1 Set up MVN
1. Open MVN Analyze/Animate
2. Open an MVN file or MVNX file or use the MVN suit in a live setup
3. Go to: Options > Network Streamer
4. Add a target destination (with default values: Host 127.0.0.1, Port 9763, Protocol UDP
5. Select a format that the UE4 Live Link plug-in will interpret properly: “Position + Orientation (Quaternion)”
6. When running UE4 on another PC, input the IP-address of that PC in the Host field. If it is the same PC as where MVN Analyze/Animate is running, you can just leave it at the default ‘127.0.0.1’ = localhost).
3.2 Set up UE4
3.2.1 Add Xsens Live Source
1. In the Project, Go to Window > Live Link to enable the Live Link Panel.
2. In the Live Link Panel Add a Source > Xsens Live Source > make sure that the Port number is the same as set up in MVN (step 3.1.4) and click Ok
The Live Link will detect and show you how many animation sources there are (correlating to MVN Systems being streamed)
3.2.2. Import Character
1. Import a rigged character into the project (you can skip step 3 if you already have a rigged character in your project you want to retarget the data to)
a. Under Mesh, make sure that you have Skeleton set to "None"
b. Under Miscellaneous, set Force Front XAsix to "Checked"
3.2.3. Create T-Pose
1. If your character is not in a TPose and facing the X-Axis you will need to do an extra step. Go to the skeletal mesh of your character and go to Create Asset > Create Animation > Current Pose, to create a new T-pose animation.
2. Name the T-pose animation and place it in a folder of your choosing (in this example Content > Mannequin > Animations). This will create a 1 frame animation with the pose you require.
3. In the Animation Timeline go to the first frame by pressing the "To Front" button
4. If required, adjust the bones of the character to represent a T-Pose and faces the X-Axis.
5. Once the character is in a TPose and facing the X-Axis, press Add Key and Apply to update the pose.
3.2.4 Retarget animation
1. In the Content Browser, locate the skeleton of your character. Right-click to open Skeleton Actions menu and choose > Xsens and either Create Remap Asset (local creation) or Create Remap Asset in Folder (to choose the folder the Asset will be placed in). This will create Remap Asset and a Data Table.
2. Open the Remap Asset just created. In the Details panel, select the T-Pose animation you just created.
2. The Target table will be set automatically. Below this table is the Remapping Assets. Unfold this to show the list of remapping options.
3. For all the available bones select the correct one from the drop-down menu. Leave unused bones on "None".
TIP: To find the segment names of your character, you can open the Skeleton that belongs to this. This can be found in the Skeleton Tree panel in the Skeletal Mesh.
4. Once all segments are filled in, press Compile and Save.
NOTE: Remap Dict may vary between characters, the example below is only applicable for the UE4 Mannequin used in this example.
3.2.5 Create Animation Blueprint
1. Now create an Animation Blueprint for your character. In the Content Browser, Right-click in the folder you want to create the “Animation Blueprint” and select “Animation > Animation Blueprint”.
2. Do not select a parent class, but do select the skeleton of your character for the target skeleton. Then click OK.
3. Name the Blueprint and double click to open it.
4. In the AnimGraph, right-click and add a “Live Link Pose”
5. Select one of the Subjects Under Live Link Subject Name (This corresponds to your MVN System)
6. Connect the Live Link Pose Node to the Output Pose Node
7. Select the Live Link Pose node just created and in the "Details" Window, you can select the retarget asset you've set up earlier (in this case: UE4_Manequin_Skeleton_Remap)
8. Now compile and save the “Animation Blueprint”. Use this Animation Blueprint in your scene.
9. Your character will now be animated live with the MVN data stream.
4 ADVANCED FEATURES
To use props, attach them to one of the bones that is in the retarget manager. For example to the left hand. The prop is retargeted in the same way as the body segments using the names Prop1 to Prop4. The props in the stream will always start at Prop1 even if you disabled prop 1 in MVN and are using prop 2.
In the T-Pose, swords have to be placed with the tip pointing towards the x-axis and guns have to be placed pointing downwards.
To set up the finger tracking, a few changes to the standard workflow have to be made. Below, the changes are listed:
- MVN Network Streamer: add "Send Finger Tracking Data"
- T-pose: Make sure the fingers are straight.
- Retargeting: include Finger bones in remap asset
4.2.1 Set up MVN Streamer:
1. In the Network Streamer Options, enable "Send Finger Tracking Data"
4.2.2. Create T-pose
1. Make sure that in the T-pose animation the finger bones are aligned with the arms.
4.2.3 Retarget animation
1. In the Remap asset (in this case UE4_Mannequin_Skeleton_Remap), include the finger segments as remap target (see example below).
After this, your character should be receiving and retargeting finger animations as well.
4.3 Packaging Project with Live Link integration
The 4.23 can be packaged with the Live Link integration. To package the project, a few changes to the standard workflow have to be made. Below, the changes are listed:
- Plugin: place directly in the Project folder
- Game Instance Class: set to LiveLinkGameInstance.
4.3.1 Install Plugin
If your MVN Live Link plugin is not installed in your project (but in your engine), you need to add it to your Project folder.
1. Go to the Unreal Engine folder. Typically this is found under:
C:\Program Files\Epic Games\UE_4.23\Engine\Plugins\Marketplace\LiveLinkMvnPlugin
2. Go to the project folder. If there is no Plugins folder, first create a new folder and name it Plugins.
3. Copy the entire LiveLinkMvnPlugin Folder to the Plugins folder of your Project.
4. If there is a Source folder in your Project folder, delete it.
4.3.2 Set Game Instance Class
1. In the project go to Edit > Project Settings > Maps & Modes.
2. Under Game Instance, set the Game Instance Class to LiveLinkGameInstance.
3. To package the project, go to File > Package Project > Windows > Windows (64-bit)
4. After building, a new folder is added to the project: WindowsNoEditor. Click the executable there to open the standalone version of the project.
NOTE: Before opening the executable, close the project first. This is because the stream only allows one port at a time and the project and executable will both use the same port at this point.
You should now have a standalone package with an executable that excepts MVN Live Stream data.