Magic Leap

Getting Started

Prerequisites

To get started with Magic Leap, you need to install Lumin SDK and Lumin Build support for Unity. To do this, follow these steps:

Go to Magic Leap Creator Portal, sign up your personal Magic Leap account following the instruction.
Download Magic Leap Package Manager from column Download. Download for Windows or Mac according to the OS you use.
Open the Magic Leap Package Manager and install the latest version of the Magic Leap Unity Package, which should be listed under Unity Packages section. You should also get the latest version of Lumin SDK in this way. If not, install Lumin SDK under Common Packages. This setup tutorial works for Lumin SDK version 0.22.0.
Double check the change list at the bottom, make sure there are Magic Leap Unity Package and Lumin SDK in the list, then click Apply.
Open Unity Hub, make sure your unity version is at least 2019.2.0f1. Click Installs and then click the 3-dot menu for the unity version you use. Click Add Modules and install Lumin OS (Magic Leap) Build Support. If you could not find Add Modules option, try downloading Unity Installer corresponding to the version you use from Unity website, and then run the installer to add the module.

Basic Setup

Create new Unity 3D Project
Go to Edit->Preferences->External Tools, set Lumin SDK path. The default path will be C:/Users/<CURRENT_USER>/MagicLeap/mlsdk/<VERSION>.
Go to File->Build Settings, select Lumin and then click Switch Platform
Go to Window->Package Manager, click Advanced and select Show preview packages, choose Yes for the warning message. Find XR Management and click install
Go to Edit->Project Settings->XR Plugin Management, select Lumin, click + then download Magic Leap Loader. Then click + again and select Magic Leap Loader under Available. Do the same for PC, Mac & Linux Standalone.
Go to Edit->Project Settings->Player, select Lumin and go to Other Settings. Under Rendering, change Color Space from Gamma to Linear. Under Identification, change Bundle Identifier to a unique name, like edu.cmu.etc.bvw19.team1. Under Configuration, change Api Compatibility Level from .Net Standard 2.0 to .Net 4.x.
Go to Edit->Project Settings->Player, select Lumin and go to Splash Image. Under Splash Screen, uncheck Show Splash Screen.
Go to Edit->Project Settings->Player, select PC, Mac & Linux Standalone and go to Other Setting. Under Rendering, uncheck Auto Graphics API for Windows, then click + to add OpenGLCore. Drag OpenGLCore and put it on top of Direct3D11. You will need to restart Editor to take effect.
Import the Magic Leap Unity Package into your project. To locate the package, open Magic Leap Package Manager and explore to Unity Packages-> Magic Leap Unity Package and you will see the folder option. Double click the .unitypackage file to import. If Unity needs to update API on importing, click I Made a Backup, Go Ahead! to continue. Now you should see [MagicLeap] folder in your Assets window.
If you receive compilation error messages like MLSpatial Audio is not found after importing the package, go to Edit->Project Settings->Player->Lumin OS->XR Settings and check "Virtual Reality Supported" to try enabling legacy XR. After a while Unity will tell you multiple XR SDK providers have been found and will disable legacy VR then (the checkbox will not be checked as well), and the compilation error should be resolved.
Now you are all set! Be ready to enjoy developing with this charming device!

Note: The Official Guide provides more details on how to setup Magic Leap for Unity projects.

Scene Setup

Remove Main Camera from Scene.
Drag Main Camera from MagicLeap/Core/Prefabs into the scene.
According to the types of function you want to use, go to Edit->Project Settings->MagicLeap->Manifest Settings, check the function you will use. Remember make sure Minimum API Level should at least be 4 and LowLatencyLightwear should always be checked. For more details, check Privileges page from official documents.

Preview Game with ML Remote Simulator

Make sure build setting has changed to Lumin Platform and the Unity renderer is set to OpenGL (with <OpenGL> on your title bar)
Go to Magic Leap->ML Remote, click Import Support Libraries
Go to Magic Leap->ML Remote, click Launch MLRemote
Click Start Simulator and after all signal lights turn green, you can hit play button in Unity Editor and see the scene in ML Remote.
Follow the instructions here to navigate and control in the simulated scene.
To stop the game, click stop button in Unity Editor. You have to stop the game first before stopping the simulator, or Unity will become suspended and may crash.
To stop the simulator, click the 3-line menu->Stop Magic Leap Remote Server.
Do not close MLRemote unless you have stopped the Remote Server, or the server will be still running in the background.
Do not close MLRemote unless you have stopped the Remote Server, or the server will be still running in the background.
Do not close MLRemote unless you have stopped the Remote Server, or the server will be still running in the background.

Note: You must launch MLRemote from Unity editor to correctly connect to the simulator when running the game.

Preview Game on Device

Make sure build setting has changed to Lumin Platform and the Unity renderer is set to OpenGL (with <OpenGL> on your title bar)
Go to Magic Leap->ML Remote, click Import Support Libraries
Go to Magic Leap->ML Remote, click Launch MLRemote
Make sure your Magic Leap device is connected to the computer. Check Connect to PC for instructions.
Click Start Device and after the signal light turns green, you can hit play button in Unity Editor and see the scene in your device.
You can debug your game as you normally do in Unity.
To stop the game, click stop button in Unity Editor. You have to stop the game first before stopping device connection, or Unity will become suspended and may crash.
To stop the device connection, click the 3-dash menu->Stop Magic Leap Remote Server.
Do not close MLRemote unless you have stopped the Remote Server, or the server will be still running in the background.
Do not close MLRemote unless you have stopped the Remote Server, or the server will be still running in the background.
Do not close MLRemote unless you have stopped the Remote Server, or the server will be still running in the background.

Note: You must launch MLRemote from Unity editor to correctly connect to the simulator when running the game.

Build and Run a Game

Make sure build setting has changed to Lumin Platform.
Make sure your Magic Leap device is connected to the computer. Check Connect to PC for instructions.
Go to Magic Leap Creator Portal, under Publish->Certificates, create your certificate. It will automatically download the privatekey.zip which contains a .privkey file. Be sure to keep the file safe. In certificate page, find the certificate you just create and download it. The certificate should be a file with extension name .cert. Important: Keep .privkey file and .cert file in the same folder. Otherwise you can't build successfully.
In Unity Editor, go to Edit->Project Settings->Player, select Lumin, and go to Publish Setting. Set the path of ML Certificate. Note that the path is an absolute path, so you need to change it when you change to another computer.
Click Build and Run and a signed .mpk package will be generated under your build folder. If you have your Magic Leap device connected, Unity will try installing this .mpk package and run it in Magic Leap. You need to accept the certificate you signed before running the application. You can also use mldb install command to install the .mpk package manually following this guide.

Note: If you have your game installed in your Magic Leap device and want to update your game, try mldb install -u command instead of mldb install.

Remove Game from Device

If you switch between development and non-development builds, uninstall the app when you're done testing each build.

On the Control, press the Home button for 3 seconds.
Select your app in the app Launcher, and then press and hold with the touchpad.
Select the trash can and press the trigger.
You can also remove it with mldb using mldb uninstall <BUNDLE_IDENTIFIER>command.

Advanced Usage

Magic Leap Device Bridge (MLDB)

Magic Leap Device Bridge (MLDB) is a command line tool provided with the Lumin SDK that allows developers to communicate with the Magic Leap One device from a PC over a USB cable or WiFi connection (see Connect to PC). MLDB provides direct access to Magic Leap device, and developers can install, run, debug and manage applications directly using this command line tool.

If you know ADB (Android Debug Bridge), you can take MLDB as the "ADB" for Magic Leap devices.

To use MLDB, you need to enable the MLDB access on your Magic Leap device:

Navigate to Settings > Device > Creator, and enable Creator Mode.
Select Enable MLDB Access to allow MLDB access.

To run commands from MLDB, open Magic Leap Package Manager, locate Lumin SDK, and click Shell to open a command line environment configured for the Lumin SDK.

A basic MLDB command is to check device connection to your computer:

mldb devices

Check this guide to explore the potentials of this powerful tool.

Resetting the device via a computer

WARNING: This method will delete all of the user information from the device!

If you manage to lock yourself out of the Magic Leap or cause the device to no longer render anything despite being on and functional, don't fret! Getting the device back up and running from initialization is straightforward and quick.

Connect the Magic Leap to one of your computer's back USB 3.0 ports using a USB-C to USB-A adapter (you must connect the device to a back USB port, as the front ones may not support the device)
If you haven't already, install the Magic Leap Package Manager
Open the Lumin SDK shell for the MLDB tool.
Then enter mldb reboot bootloader. If your device is locked, then it will take 60 seconds for it to reboot. The bootloader is used for fastboot commands, which we need for doing a hard reset.
Enter fastboot erase userdata. This will erase all of the user information from the device, including any improper configuration information and any private access locks.
Lastly, enter fastboot reboot . This will reboot the device into it's typical startup state.

That's all you need to do! You can now go through the initialization process to sign into the device and set up the locking information on it again.

Connect to PC

This tutorial only works for Magic Leap with OS 0.97 or higher.
Open Magic Leap Package Manager. Under Common Packages, find Device Stream and install it.
Launch Device Stream.
Use USB-A 3.0 to USB-C cable to plug your Magic Leap device to the computer.
Turn on Magic Leap device, accept USB connection in Magic Leap and accept the connection request of Device Stream, then you should see the device identifier on Device Stream with green signal light and streaming display.
You can also connect Magic Leap using Wi-Fi. Make sure your device has Internet connection. Go to Device Stream and click the reverse triangle button next to device identifier and then click the gear menu icon. Click Use Network, and you need to accept the connection request again in Magic Leap. Then you should see the ip address of the device on Device Stream and see the streaming display. Note that you don't need to use the USB cable after the device is connected via Wi-Fi, but you need to use the cable to reconnect your device after device restart.
To check the connection status. Open Magic Leap Package Manager and explore to Lumin SDK, click Shell and use mldb devicescommand to check.

Streaming to Device

To PC

The Magic allows for streaming of headset on Device Stream on OS 0.97.

Make sure you have Device Stream installed. If not, open Magic Leap Package Manager. Under Common Packages, find Device Stream and install it.
Connect your Magic Leap device to PC. For detail instruction, check Connect To PC.
You will see the stream display in the display window of Device Stream.
Don't use the recording function inside Device Stream now, no audio will be recorded in the current beta version. To record the video, check Recording Video for more information.

To Android or iOS

The Magic Leap allows for streaming of headset visuals to a mobile device as on OS 0.95.

Download the Magic Leap companion app from the iOS or Android store to mobile device
Make sure Bluetooth is enabled on mobile device
Open the Magic Leap Companion App and move to the Magic Leap Headset
On the Headset, enter the settings Device Settings menu.
Under "Creator", make sure that Enable Device Streaming is checked affirmative
Under Connection Settings, enable Bluetooth and pair to your mobile device
Under Controllers, scroll down to Scan for Companion App
Connect and pair to mobile device there as well.
You will be asked to pair/accept connections on your mobile device as well as on the Magic Leap. Accept them.
On the Companion App, make sure that "waiting to connect" now displays "connected'
On the sidebar menu, scroll and select to Device Streaming (beta).
Magic Leap will pop up a notification to allow device streaming. Allow the device to stream.

Your device should now show a stream of the Magic Leap's view.

Note: Device Streaming and Device Recording/Screenshot can not be used at the same time (as of 0.95)
To record the stream, use the mobile device's built in screen recording software
- In iOS you must start a screen recording first before connecting to Device Streaming. This is because leaving the Companion App while streaming breaks the stream.

Recording Video

Please download the recording script from here and follow the instructions. The script assumes that you have Lumin SDK v0.22 installed in the default location (C:/Users/xxx/MagicLeap). This script utilizes mldb capture video command to record the video.

You could also record video using Device Stream. Don't use the recording function of Device Stream (audio will be lost), instead, use OBS and treat Device Stream as normal video output to record.

Programming Information

Placing Your Scene

The Magic Leap Unity Plugin is shipped with a component [Placement] that can help place your scene or any objects aligned with world mesh. You can check the scene [Placement] in the plugin's examples to learn how to do so.

Unity API Reference

You can find the Unity API Reference here, or you can install it on your local PC following these steps:

Open Magic Leap Package Manager.
Open Unity API Documentation tab.
On the right side panel, download the latest version.
On the left panel in the bottom, "Apply".
The right hand panel now shows a button that opens the reference when you click on it.

Single Pass Instanced Rendering

Magic Leap uses single pass instanced rendering by default. It may introduce some problems to customized shaders and only the left eye will render the correct result if the shaders lack the support of this mode. You need to manually modify the shader and related scripts to make sure the camera poses are correct for each eye and to support instanced rendering. Please check Unity's reference for more information.

If you have no idea how to work correctly with single pass instanced rendering, the easier way is to fall back into muti pass rendering with some performance lost. You can turn it on through [Edit -> Project Settings -> XR Plugin Management -> Magic Leap Settings] and enable [Force Multipass] option.

Art (2D & 3D) Information

The Official Art & Anmiation Guide provides best practices for artists.

Sound Information

The Unity Magic Leap plugin is shipped with Magic Soundfield Audio (MSA) plugin that supports spatial audio in Magic Leap. Check this guide to learn how to set up MSA in Unity.

The Official Sound Design Guide provides best practices for sound design of Magic Leap Apps.

Design Guidelines & Tips

The Official Design Guide is a good starting point to learn how to design a game or an immersive experience for Magic Leap.

The near clip plane distance of Magic Leap One is 37cm (14.57 inches). Hand interaction within this distance should be avoided.
Magic Leap One has a horizontal FOV of 40 degrees, a vertical FOV of 30 degrees, and a diagonal FOV of 50 degrees. The FOV range is a bit narrow. Considering using Vignette effect and guiding arrows when you are designing a 360 degree scene.

Example Projects

We provides a template project configured with Magic Leap Unity Package 0.22.0.

To set up for this template project, follow the steps below:

Make sure you have finished all the steps in the Prerequisites section first.
Extract the zip file and rename the extracted MagicLeapTemplate folder to your team name (e.g. Team01).
Open Unity Hub, click Add and select this folder. You will find this project in the Projects list. Click this project to launch Unity editor.
Go to Edit->Preferences->External Tools, set Lumin SDK path. The default path will be C:/Users/<CURRENT_USER>/MagicLeap/mlsdk/<VERSION>.
Go to File->Build Settings, select Lumin and then click Switch Platform.
Go to Edit->Project Settings->Player, select Lumin and go to Splash Image. Under Splash Screen, uncheck Show Splash Screen.
Go to Edit->Project Settings->Player. Change Product Name to your team name (e.g. Team01). Then select Lumin settings tab below and go to Other Settings. Under Identification, change Bundle Identifier to a unique name, like com.bvw19.team1. This is the unique name for the mpk package that you will build.
If you are using MLRemote for debug, don't forget Menu->MagicLeap->ML Remote->Import Support Libraries.
If you are using Perforce, configure your Perforce settings in Edit->Project Settings->Editor.
Now you are all set! Example scenes are located in Assets/BVW/SampleScenes. Be ready to enjoy developing with this charming device!

Previous ETC Projects

https://www.etc.cmu.edu/projects/artales/

BVW

Currently there is no BVW-specific information to know about Magic Leap.