- A Windows machine with Kinect Studio Installed

Fastmocap is a motion capture technology, that enables the 3D artists to create natural and organic character animations in a very fast and easy way, taking advantage of the Microsoft Kinect Sensor. Fastmocap is a motion capture technology, that enables the users to create natural and organic character animations in a very fast and easy way, taking advantage of the Microsoft Kinect Sensor. You can choose between the basic and the pro version depending on the features you would like to have.

- Kinect v1 or Kinect v2

Motion Capture With Kinect

- A mobile VR set (Google Cardboard or Samsung Gear VR)

- Kinect SDK for Windows

Kinect Motion Capture Software Mac Download

- Unity

- Android Studio or Xcode

- Cardboard SDK (detailed tutorial for iPhone and Android)

- NodeJs (for networking)

Simple Setup Guide (more detailed tutorial in the works)

After unzipping the download, open the KinectVR-broadcaster folder as a Unity Project

Open the broadcast unity scene

Select AliceManager inside the Kinect prefab, change the HOST to your local IP

Export as a Windows Standalone build, this will broadcast raw Kinect data to node.js

Right click the console shortcut, and select it's properties

Copy the url to the js file in the target field and make sure node.js is installed

Right click and edit the KinectVRServer.js file and modify var HOST to your local IP

Open the KinectVR-Server shortcut or run the KinectVRServer.js file in node.js

Open the KinectVR-broadcaster you exported earlier, it should auto connect to node.js

Open the KinectVR folder as a Unity project

Open the KinectVR-Client scene

Open the KinectVR prefab to find AliceManger

Edit the Host IP inside the AliceManager

When you press play, Unity will connect to node.js and recieve the Kinect data

by Andy Jeong, Yue Wang, Professor Mili Shah (Advisor)

Abstract

Body joint estimation from a single vision system poses limitations in the cases of occlusion and illumination changes, while current motion capture (MOCAP) systems may be expensive. This synchronized Azure Kinect-based MOCAP system serves as a remedy to both of these problems by creating a low-cost, portable, accurate body-tracking system.

Keywords: Motion capture (MOCAP) system, synchronization, Kinect, body-tracking

See Azure.com/Kinect for device info and available documentation.

Link to submitted poster to ACM SIGGRAPH’20: Poster

Link to submitted abstract to ACM SIGGRAPH’20: Abstract

(Received 4 feedback; 3 neutral, 1 slight negative)

Link to documentation: Documentation

Demo

Check out the outcomes on some various movements!Note: this demo experiences some offset due to a parallax problem (devices are at a lower height than the human).

Overview

Flowchart

System Setup

Hardware

• Ubuntu 18.04 / Windows PC machine with USB3.0+ support
• USB Hub for multi-device connection (Targus 4-Port USB 3.0): Link
• USB 3.0 Extension Cable for multi-device connection: Link
• Audio Jack Cables for multi-device connection: Link

Software

• Azure Kinect Senor SDK (K4A) (-lk4a)
• Azure Kinect Body Tracking SDK (K4ABT) (-lk4abt)
• OpenCV (`pkg-config --cflags --libs opencv`)

Building

Test Setup

Configuration

• Daisy-chain configuration: supports connection of 2+ subordinate mode devices to a single master mode device (RS-232)

Testing Environment

Camera Calibration to capture synchronous images

• reference: green screen example from Azure Kinect SDK examples on its GitHub repository

Outcomes

With multiple devices in place, joint estimation is still performed as if there is no occlusion or lighting effect.The following videos and images are tested in the test setup shown above.

Videos Samples

Kinect Motion Capture

Synchronization

• on the right: joint angles for angles designated as below

Occlusion / Illumination Effect Verification with 3-Device System

Example of selection of data streams by confidence levels per joint

Azure Kinect SDK Details

Azure Kinect SDK is a cross platform (Linux and Windows) user mode SDK to read data from your Azure Kinect device.

The Azure Kinect SDK enables you to get the most out of your Azure Kinect camera. Features include:

• Depth camera access
• RGB camera access and control (e.g. exposure and white balance)
• Motion sensor (gyroscope and accelerometer) access
• Synchronized Depth-RGB camera streaming with configurable delay between cameras
• External device synchronization control with configurable delay offset between devices
• Camera frame meta-data access for image resolution, timestamp and temperature
• Device calibration data access

Current Work

1. Gait Analysis on Exoskeletons

OpenPose, AlphPose, Kinect, Vicon MOCAP system

2. Graphical Visualization of Tracked Body Joints

Media art collaboration

3. Drone Movement Synchronzation from Human Pose

Control of drone system (crazyflie)