Microsoft Kinect - Open NI

From Unity3D

Jump to: navigation, search

Note: "Documentation" sections are articles that tell of the problems, solutions, and methods behind implementing a piece of hardware into Unity.


Contents

Process

  1. Discovered a simple method for implementing the Kinect into Unity.
  2. Downloaded all side programs (OpenNI, NITE, Kinect drivers, OpenNI hack) and found all versions that work with our machines.
  3. Tested sample code and then handed it off to be revamped for simple model swapping and raw output of Kinect data.


How to Install Drivers and OpenNI for Microsoft Kinect

  1. Download Kinect_Install_Files.zip
  2. Unzip the folder and begin Installing OpenNI-Bin-Win32-v1.0.0.25.exe
  3. After OpenNI finishes, Open the NITE SDK Installer (NITE-Win32-1.3.0.17.exe)
  4. Before the installation begins, make sure to enter the following Free License key from Openni.org. It should be the last step before it begins

License Key: 0KOIk2JeIBYClPWVnMoRKn5cdY4=

  1. Unzip the SensorKinect file (avin2-SensorKinect-0124bd2.zip)
  2. Connect the Kinect device to your PC
  3. Open the Device Manager (Start > Run... > devmgmt.msc) and verify that 'Xbox NUI Motor' is on the list (but with an exclamation mark)
  4. Right-click on the device and choose 'Update Driver.' Make windows search for a driver in the avin2-SensorKinect-0124bd2\Platform\Win32\Driver folder.
  5. There should now be three devices. 'Kinect Motor' and 'Kinect Camera' should be successfully installed. The third device will fail, which is okay. This is 'Kinect NUI Audio.' This is the driver for the array mics, but currently we don't have access to it.
  6. Install the executable (SensorKienct-Win32-5.0.0.exe) located in avin2-SensorKinect-0124bd2\Bin
  7. Run C:\Program Files(x86)\OpenNI\Samples\Bin\Release\NiUserTracker.exe to test if everything works. If this fails, try to run C:\Program Files\OpenNI\Tools\vcredist_x86.exe


Adding the DLLs and Scripts

  1. Make sure all the drivers are installed before attempting to use.
  2. If you haven't already made your project, do so.
  3. Inside the Kinect_Install_Files folder, open Project Files.
  4. Copy both the UnityInterface.dll and OpenNI.xml files into the root of your project folder.
  5. Copy the scripts contained inside the Scripts folder into your project.

The source for the DLL originally came from the now defunct PrimeSense. It is available here: [1].

Setting Up the Scripts - Quick Overview

There are three total scripts that are required to use the Kinect inside of Unity: KinectWrapper, AvatarController, and KinectManager. Make sure each of these scripts are in the same folder, otherwise you may have pathing issues.


KinectWrapper wraps the functions form UnityInterface.dll and allows them to be called in C# / Unity.

  • This script just needs to be present within your project. You do not need to attach it to an object within the scene.


AvatarController contains all of the methods required to map your avatar to the Kinect.

  • First, attach the script to the model you want to control.
  • Next, Drag and drop the appropriate bones to be controlled by the Kinect skeleton.
  • Afterwards, check the flags for whether the model should move, how fast, and whether the movement should be mirrored.
  • You can define which player controls which model in the KinectManager.


KinectManager has all of the update logic required to use the Kinect.

  • Attach this to the Main Camera.
  • Add all of the models you want to control to the appropriate list for the appropriate players.


Using AvatarController

Attaching the Bones

To set up the AvatarController with your model, attach the script to the model you want to control.

Next, select the model. Expand it, and select the skeleton. Within the skeleton, you will find all the appropriate bones you need to control your model. Drag and drop the appropriate bone to the bone you wish to control. (See Figure 1)
Fig. 1, Avatar Controller

Note that you can attach as many or as few bones as you want to the model - it will leave those bones unmodified if you don't pair them to the AvatarController.

Also, please note that the Avatar needs to have its rotation set to 0 across X, Y, and Z. If you want to have model rotate, make sure to set an Offset Node and rotate that instead (see "Rotating the Character"). This is due to the fact we are using Quaternion coordinates.

The "Mirrored Movement" flag causes the model to move the opposite limbs (Left Arm -> Right Arm and vice versa).

WARNING: Be careful when rescaling the model - this can cause the rotations and movement rate to behave oddly.


Moving the Character

To have the character move in space, check the "Moves in Space" box within the editor. To apply vertical movement, check "Vertical Movement."

To change the speed of the character, raise the movement rate. The characters speed is multiplied by .001f (Unity's framerate), and the Movement Rate field multiplies that.


Rotating the Character

To have the character rotate, you must first set an Offset Node. This is done by creating an empty Game Object and parenting the model to it. (See Figure 2)
Fig. 2, Parenting to NinaController
Then, in the model, make sure to Drag and Drop the offset node into "Offset Node". (See Figure 3)
Fig. 3, Offset Node

After this is done, you can rotate the Offset Node to move the character.

WARNING: The current version has problems with rotations on the Y and Z axis. This may be fixed by the time you read this, however.

Samples and Install

Below are Installation files and demos for Kinect
Kinect Install Files
Kinect Example Projects


Youtube Videos:
Kinect Simple Demo
Kinect Show Demo

Vimeo Videos:
Kinect Simple Demo
Kinect Show Demo

Personal tools