Kinect 360

I’m currently adapting a Kinect 360 to stream skeleton data over a websocket connection so that it can be used by my gearVR for Full Body Tracking in VR. Responsive enough Full Body Tracking combined with a type of haptic stimuli input will allow for experimentation with the psychological phenomena of the Body Transfer Illusion. Currently stuck at converting skeleton data to proper Mecanim transforms.

Hardware in use
HMD: Samsung Gear VR with a Galaxy S7
Positional Tracking: NoloVR
Body Tracking: Microsoft Kinect Sensor for XBox 360

Software in use
Unity 2017
Adaptation of Kinect with MS-SDK Unity Asset
HomeBrew Kinect Skeleton Streamer

Here is an example of the Skeletal Data being directly mapped to cube transforms.

Here is an example of Skeletal Data being mapped to Mecanim Model’s transforms.

We’re not quite there yet. Currently taking a break on the project, but will release the streaming software and Unity code on my github when finished.

Note: Microsoft is iffy about the scenarios it lets you run the Kinect 360 on a PC. I’ve noticed off a fresh reboot I can get away with running the Skeleton Demo forever without issue, but if I happen to be running Unity in the background or if I’m running code that has a namespace that doesn’t start with Microsoft.Kinect.* at the very least will cause the Kinect sensor to timeout / cease to work after (1:30) 90 seconds.