Difference between revisions of "HTC Vive"

From ETC Public Wiki
Jump to: navigation, search
(Added note of role conflict issue with trackers)
m (Fixed typo in skeleton blend script)
Line 210: Line 210:
  
 
// Blend to full skeleton tracking within 0.5 seconds
 
// Blend to full skeleton tracking within 0.5 seconds
rightHand.BlendToSkeleton(poser, 0.5f);
+
rightHand.BlendToSkeleton(0.5f);
  
 
// Blend to the current animation specified by the hand skeleton's animator within 0.5 seconds
 
// Blend to the current animation specified by the hand skeleton's animator within 0.5 seconds

Revision as of 01:14, 16 October 2019

HTC Vive is a virtual reality headset that uses "roomscale" tracking that allows the player to move in 3D space. It uses motion-tracked handheld controllers to interact with the environment. As a part of the BVW course, students will be using HTC VIVE Tracker VR Object Tracker and Vive Controllers.

Lighthouse & Device Compatibility

Here is a matrix showing compatibilities for all the known devices from Vive and Index family:

Vive Headset Vive Controller Vive Tracker (Grey Vive button) Vive Pro Headset Vive Pro Controller Vive Tracker 2018 (Blue Vive button) Index Headset Index Controller
Lighthouse 1.0 Supported Supported Supported Supported Supported Supported Supported Supported
Lighthouse 2.0 Not Supported Not Supported Not Supported Supported Supported Supported Supported Supported
  • One headset supports pairing two devices (in most cases are controllers). One dongle supports pairing one device additionally. Using more than 2 dongles are not as stable as expected and will be difficult to pair devices correctly at the first attempt.
  • Sometimes you will fail to find the speaker for Index Headset with no audio output as well. You will need to go to Sound Control Panel, select Index HMP in Playback devices, right click on it and enable it. You might need to restart SteamVR and headset if it is still marked as Disabled.

Getting Started

Basic setup

  1. Open Unity Project
  2. Go to Edit -> Project Settings -> Player
  3. Click on XR Settings in the Inspector
  4. Check Virtual Reality Supported
  5. Remove Oculus if it exists in the list under Virtual Reality SDKs (This can be done by selecting Oculus and hitting the minus sign at the bottom right)
  6. Make sure OpenVR has been added under Virtual Reality SDKs (If you don't see it, hit the plus sign at the bottom right and select it from the list)
  7. Download Steam VR plugin from Unity Asset Store
  8. Import it into your project, and click on "I Made A Backup", and click on "Accept All"

If you are using Steam VR plugin 2, these extra steps are required:

  1. Go to Window -> SteamVR Input.
  2. If you open this window for the first time, it will ask you to copy example action jsons. Click on Yes to continue.
  3. This opens up a SteamVR Input window. Click on [Save and Generate] and SteamVR plugin will generate action config json files (under project root folder before v2.4 and under Assets/StreamingAssets/SteamVR in v2.4 or newer versions) and action set classes (under Assets/SteamVR_Input).
  4. Close the SteamVR Input window.

Scene Setup

  1. After import is complete, delete [Main Camera] from the hierarchy
  2. From your project window, select Assets -> SteamVR -> Prefabs
  3. Drag [CameraRig] into your hierarchy
  4. Now you can get tracking information from controllers!

Tutorial Links

  1. Unity VR Overview
  2. HTC Vive Tutorial
  3. SteamVR Plugin Input Overview
  4. Input for OpenVR Controllers
  5. Skeleton Poser Tutorial

Programming Information

Binding Actions (SteamVR Plugin 2)

SteamVR introduces a new action based input system that define an action as an abstract type of input (like input axis/buttons in Unity's InputModule). All the actions can be bound to different buttons in different applications and on different devices like Vive controller, Index controller, etc. These binding settings are saved to a binding file (bindings_xxxx.json). All the actions are defined in actions.json. Developers need to include these binding files when publishing the application as the action sets and the default bindings for their applications.

A new Unity application will have default action bindings generated from the first click of [Save and Generate] in SteamVR Input window. Developers can either modify default actions and bindings, or create new actions in SteamVR Input Window.

Create New Actions

  1. Go to Window -> SteamVR Input Window.
  2. Select an action set (like "default"), and click "+" icon in the "In" panel (for input) or "Out" panel (for output, only haptics for the moment).
  3. Click [Save and Generate] to apply new action sets and update generated code files.

Binding Actions

Note: You need to bind actions on a computer that has Vive headset connected and controllers paired.

  1. Go to Window -> SteamVR Input WIndow.
  2. Click [Open binding UI] and a binding configuration webpage will appear. (The appearance might be not correct on the Edge browser, but the page works well). If there is no webpage available, open SteamVR application and click [Menu -> Devices -> Controller Settings -> Edit Controller Binding -> Show More Applications], and then select your application and edit the current binding.
  3. Modify your action bindings:
    1. Switch to an action set page (in most cases should be "default").
    2. For buttons (Boolean): Click '+' Icon of the controller button you want to bind, and then select [Use as Button].
    3. For axises (Single): Click '+' Icon of the trigger button, and then select [Use as Trigger].
    4. Click [None] of the newly added binding item, and then select action name in the pop-up window.
    5. Finally click '✔' tick icon to apply.
  4. Click [Replace default binding] so that the binding json files under your project folder will be updated.
  5. Go back to binding selection page and select your official binding and click [View], then click [Select this binding] button to apply this binding. Do not use local binding during the development phase because local binding changes will not be reflected on binding json files under your project folder.

Using Vive Controllers

Here are some code snippets to get input from Vive controllers:

// Usage 1
SteamVR_Action_Boolean action = SteamVR_Input.GetBooleanAction(actionName);
bool state = action.GetState(SteamVR_Input_Sources.inputSource);

// Usage 2
SteamVR_Input.GetState(actionName, SteamVR_Input_Sources.inputSource);

// Usage 3
bool state = SteamVR_Actions._actionSetName.actionName[SteamVR_Input_Sources.inputSource].state;

// Usage 4
bool state = SteamVR_Actions.actionSetName_actionName[SteamVR_Input_Sources.inputSource].state;

Other types of actions (single/float, vector/axis, haptics, etc.) can follow the same ways as described above.

In addition, the SteamVR Plugin provides a more comprehensive interaction system that supports high-level interactions like grab, attach, teleport, etc. Check the documentation for more details.

Archive:

Note: The following content only applies to SteamVR Plugin 1. If you are using SteamVR Plugin 2 or above, the code below will not work.

See the commented code snippet below

// get a reference to the Vive controller
SteamVR_Controller.Device myController = SteamVR_Controller.Input(index);

// then you can get inputs from the controller by calling methods on it
bool isHairTriggerDown = myController.GetHairTriggerDown();
bool isHairTriggerUp = myController.GetHairTriggerUp();

// you are encouraged to look into the "Device" script to see what kind of functionalities it provides
// so you might find things like this
bool isGripDown = myController.GetPressDown(EVRButtonId.k_EButton_Grip);
Vector2 touchPad = myController.GetAxis(EVRButtonId.k_EButton_SteamVR_Touchpad);

Using Index (Knuckles) Controllers

Binding Actions

The action binding process for the Knuckles controller remains the same as the Vive controllers. You need to create actions and bind actions to input buttons or axes from the controllers. Remember to switch to Index controller in the binding UI page before you bind the actions.

The only difference is that a lot of new input buttons/axes have been added and you are allowed to bind skeletons as well (Skeleton also works for other controllers, but it is mainly designed for Knuckles controller). We recommend you to take a look at this link to learn what the new input variables are. Major changes include A/B buttons, Grip Force Sensor (with two axes Pull and Force), Trackpad (with a two-axis Position and a Force axis) and Thumb Stick (with a two-axis Position). The Touchpad and the Grip, Menu buttons in Vive controllers are not available as well.

Knuckles overview.png

Using Skeleton Input

SteamVR introduces an action type called Skeleton to provide skeleton input simulated from different controllers. The Knuckles controllers are designed to fit with hands and thus provide an accurate skeleton input that can track the movement of each finger (thumb by the Trackpad, index by the Trigger, middle, ring and pinky by the Grip Force Sensor, also called Grip FSR).

The skeleton input data can be applied to a skeleton defined by a root transform in Unity, using [SteamVR_Behaviour_Skeleton] component. The Unity SteamVR plugin is shipped with some example hand prefabs (located in Assets/SteamVR/Prefabs) for you to explore how this feature works. You can get more input data like finger curls (thumbCurl, indexCurl, etc.), finger splays (thumbIndexSplay, indexMiddleSplay, etc.) and the the transform of each joint (root, wrist, etc.) from the skeleton action or the behaviour component.

Here is an example that detects the scissor gesture:

SteamVR_Action_Skeleton skeletonAction = SteamVR_Input.GetSkeletonAction("SkeletonRightHand");

// Or get the action in this way
SteamVR_Behaviour_Skeleton rightHand = GetHandComponentFromSomewhere();
SteamVR_Action_Skeleton skeletonAction2 = rightHand.skeletonAction;

bool isScissor = skeletonAction.thumbCurl > 0.1 &&
                 skeletonAction.indexCurl < 0.1 &&
                 skeletonAction.middleCurl < 0.1 &&
                 skeletonAction.ringCurl > 0.9 &&
                 skeletonAction.pinkyCurl > 0.9;

If you wish to create your own hand model, please follow the naming conventions and the hierarchies of all the skeleton joints in the example prefabs to avoid mismatching problems. The Hand Skeleton guide for OpenVR SDK explains more on how to find the built-in hand models and what each joint means.

More information of [SteamVR_Behaviour_Skeleton] component and [SteamVR_Action_Skeleton] input can be found in this link. We highly recommend you to check the link first before getting started with this system.

The Skeleton Input is usually used together with Interaction System and Skeleton Poser that will be introduced below.

Note: If you have disabled or removed two controller pose objects inside [CameraRig] and try to use [SteamVR_Behaviour_Skeleton], you will find SteamVR fails to track everything correctly because it's not initialized. A workaround for the moment is to add SteamVR.Initialize(); anywhere in the Start method of a MonoBehaviour script and attach it to the game scene. An easier solution is just to keep the controller objects but remove the controller models inside instead.

Using Skeleton Poser

The Skeleton Poser serves as a pose driving system for hand skeleton on interactable objects. Usually each object will be bound to a different pose when it is interacting with the hand, so a system that allows to reuse poses for different objects will help, and that's why this system was proposed.

A [SteamVR_Skeleton_Poser] component is usually attached to an interactable object and it supports one main pose and a blending mechanism between multiple hand poses for one object. The editor interface of the [SteamVR_Skeleton_Poser] is divided into Pose Editor and Blend Editor. In most cases you will only need to set up the main pose in Pose Editor.

The Pose Editor allows you to define several poses for the current object while the first pose is always the main pose that will be used by default. You can create a new pose or select an existing pose for each tab. You can also reset the pose to a reference pose with RESET TO REFERENCE POSE button.

A pose is defined by the transforms of each bone (joint) of the hand skeleton (rig). You can click on Left Hand or Right Hand icons in the editor area to show the current pose for one hand and start editing the pose by modifying the transform of each hand bone. You can also set whether each finger should be blended to this pose through finger movement options for each hand. The option Static and Free are self explained, and Extend means you can extend your finger even if the pose of the finger is defined as contracted. The option Contract is exactly the opposite of the Extend option.

When you finish editing the current pose, you can click on Copy Left(Right) Pose to Right(Left) Hand to mirror the pose on the other hand. Then don't forget to Save Pose and click on Hand icons again to hide the preview hands. If you are using a prefab these temporary hand objects should never appear when you save the prefab changes.

The Blend Editor allows you to blend between multiple poses defined in the Pose Editor if you wish to add a second layer of hand pose upon the the main pose. Usually the blend behaviour is triggered by an Action or scripts like poser.SetBlendingBehaviourValue(name, value) and poser.SetBlendingBehaviourEnabled(name, enabled). You can refer to the source code and the official manual to learn more about this feature.

The Skeleton Poser is mostly used together with SteamVR's Interaction System because the Interactable script will activate the pose automatically when the object is attached to the hand. Remember to disable HideHandOnAttach and enable HideControllerOnAttach if you are using this script. If you wish to activate the pose with your own code, here is an example on how to do so:

// Get the behaviour skeleton component from somewhere
SteamVR_Behaviour_Skeleton rightHand = GetHandComponentFromSomewhere();

// Blend to a SkeletonPoser within 0.5 seconds
rightHand.BlendToPoser(poser, 0.5f);

// Blend to full skeleton tracking within 0.5 seconds
rightHand.BlendToSkeleton(0.5f);

// Blend to the current animation specified by the hand skeleton's animator within 0.5 seconds
rightHand.BlendToAnimation(0.5f);

We highly recommend you to go through the official Skeleton Poser tutorial, especially the video version, to get a better understanding of this system. You are also encouraged to check the Skeleton Poser manual to learn more. Additionally, check out the examples in the Interaction System first so that you will have a better idea on how to make good use of this feature.

Note: The recent SteamVR plugin (v2.4.5) has some issues on the editor interface of the [SteamVR_Skeleton_Poser] component so that it will be displayed incorrectly. We recommend you to apply a temporary fix according to this GitHub issue before it's fixed.

Using Vive Trackers

Note: When you are using trackers with controllers, make sure to set the tracker roles to anything other than Left Hand and Right Hand (described in Method 2 below) to avoid potential conflicts with controllers!

Method 1

  1. Import TrackedObject Script from here (This script only works for at most four trackers, which is the requirement for BVW Round 1/2)
  2. Create an empty GameObject (or, to be able to see it in the game, you can create a sphere)
  3. Add "TrackedObject" component to it, and set the component's TrackedDevice to "Tracker1" or "Tracker2" or other options
  4. Each device correspond to a tracker but the mapping of tracked GameObject to physical trackers may not remain the same. If you hope to always get the same mapping of physical trackers, consider Method 2 below.

In play mode, tracking information will be send to the "SteamVR_TrackedObject" component, and will be used to modify the position and rotation of that GameObject. Trackers that are paired but not tracked/powered on will be marked as [Untracked] in the Hierarchy window in the Unity editor.

Archive:

  1. The old TrackedObject script can be downloaded here if you are using SteamVR Plugin 1. Note that when using this old version, you have to turn on and set up all the trackers correctly before you run your game. It will not automatically update newly detected trackers when the game is running. It will also fail to update the trackers correctly when you are working with multiple scenes.

Method 2

WARNING: The current SteamVR runtime fails to have stable support for tracker pose bindings so we do not recommend this method for the moment. Use it at your own risk.

SteamVR Plugin 2 introduces a special type of action called Pose, which maps to a transform (position and rotation) inside Unity engine. Each device (including trackers) can be bound to a pose. You can bind a pose with [SteamVR_Behaviour_Pose] script to gain the ability of tracking transforms of the specific device automatically.

This script requires a property called Input Source to distinguish between different devices. Fortunately, you can assign each tracker to a different input source (also called role) in SteamVR so that each time you run your game, the trackers' mapping to your gameobjects will not change.

Here are steps to assign roles for trackers. Note that you need to have trackers paired on your computer first.

  1. Click SteamVR Menu -> Devices -> Manage Vive Trackers.
  2. Select the box right after "Select Role" label, which is usually "Held in Hand".
  3. Select the role in the pop-up window. Then the new role will be assigned to this tracker.
  4. Set the tracker bindings to the pose actions. You can watch this video for instructions.

Notes and tips:

  1. Rendering the tracker: Create a child GameObject under the tracked GameObject, add "SteamVR_RenderModel" component. In that component, set "ModelOverride" to "{htc}vr_tracker_vive_1_0"
  2. To get the best tracking quality, make sure the tracker is always visible to at least one base station
  3. If you see poor render quality, make sure you are not in Steam Menu in the headset. If you are in the menu, press the menu button on the vive controller to exit
  4. If the LED is not green on the tracker, press and hold the button for a few seconds until it turns blue and then green
  5. If the LED is blinking, it means it is pairing with the headset or the dongle

Testing Vive Worlds

  1. When you are in the Bullpen, the easiest way to test your Vive world is using an FPS controller instead of the Camera Rig.
  2. However, make sure that you leave enough time to test in the platform room with actual Vive.
  3. If you are using SteamVR Plugin 2, you can use the [Player] prefab from SteamVR Interaction System and switch to 2D mode to debug.

Stereo Rendering

Stereo rendering may introduce some problems to shaders that use render texture and camera position. In most cases you need to manually modify the shader and related scripts to make sure the camera poses are correct for each eye. Please check Unity's reference and this post for more information.

Art (2D & 3D) Information

As with any stereo device, rendering performance must always be taken into account, as the image is being drawn twice for each eye. Make sure that your models are sufficiently low-poly, and that any custom shaders are not taxing on the device, to maintain a high refresh rate and provide an optimal user experience.

Sound Information

It is highly recommended to use spatial audio to enhance immersive experiences in Vive applications.

HTC also introduces Vive 3DSP plugin for Vive Pro that can be integrated into Unity to enhance audio spatialization in VR.

Design Guidelines & Tips

  • Make sure the props you use do not block the trackers, it will lose tracking
  • It is not a good idea to have players hold the tracker directly in their hand, try using props/gloves and attach the tracker to it
  • Try to use the roomspace (marked by white tape on the floor)
  • If you're disabling the controller models and using your own models (hands, sword etc.), make sure they are placed at a good distance from your body so that it feels natural to the player.
  • Try using spatial sound in VR to take advantage of the roomspace

Example Projects

Currently there are no example projects for HTC Vive.

Previous ETC Projects

  1. https://www.youtube.com/watch?v=Qs154zvFjb0
  2. https://www.youtube.com/watch?v=OGYf13tbx-Y
  3. https://www.youtube.com/watch?v=vUyqlNyJJis
  4. https://www.youtube.com/watch?v=c76v5xrVUfE

BVW

For Round 1 and Round 2 in the BVW course, students are required to use Vive Trackers in their worlds.