Eyegaze (Eye Tracking)

From ETC Public Wiki
Jump to: navigation, search

What is the Eyegaze System?


The Eyegaze system is an eye tracking system of four cameras (two external head tracking cams and two internal eye tracking cams on motorized arms) that can follow a user's head in space and track the position on the screen that they are looking at, running at 60fps and with no equipment or accessories the user would need to wear. The only requirement is that the intended user must perform a short calibration sequence before using any eye tracking enabled application.

In practice, the Unity wrapper functions very similarly to controlling a mouse pointer with your eyes, and in fact the emulator uses mouse position to simulate gaze position.

Using the Eye Tracking Wrapper


Always start your application from the Calibration scene, which contains the only instance of the EyeTracker prefab. That prefab starts up the eye tracking hardware and runs the calibration procedure. Eye tracking must be started at most once per application, so the EyeTracker prefab is flagged as DontDestroyOnLoad(true). The prefab should never be made a child of an object that is not also flagged as DontDestroyOnLoad. You may edit the calibration scene to fit your experience, but the logic should remain the same: a splash screen of some sort that waits for calibration to run, then some sort of user input to transition to the main scene after calibration is complete.

Eye Tracker Prefab Values

  • Horizontal and Vertical Resolution - The eye tracking system has a 1080p Asus monitor mounted to it, so these should almost always be set to 1920 and 1080. If developing from another machine and then testing on the Eyegaze unit later make sure you update these values accordingly. Screen.width and Screen.height from within Unity return the Game subwindow's size, not the full screen resolution, and Eyegaze needs these values to be full screen dimensions, so that will not work as a shortcut.
  • Smooth Buffer Size - The higher this value the smoother the gaze point will move, but higher values will introduce a little bit of delay. Default is 12, min is 2, max is 20.
  • Gaze Point X and Y - Last known good gaze coordinates. If the eye tracker loses track of the user's eyes, these values will stay at the last spot it saw the user looking at.
  • Pupil Diameter - Ideally this would return the pupil size in millimeters, but for some reason the only value coming out of the eye tracker for this is 0.0f. Will need to contact Eyegaze to find out more. - CURRENTLY NOT WORKING
  • Found Gaze - A bool saying whether or not the eye tracker can see the user's eyes.
  • Use Emulator - Sets the prefab to use the mouse position to simulate the gaze position. While F2 is held down Found Gaze gets set to false, otherwise it will always be true.

Making Builds using the Eye Tracker

Standalone builds should be placed in the Builds folder of the base project folder. It has all of the DLLs and other files and folders needed for a standalone build to function.

  • Currently all of the DLLs from the C:\Eyegaze folder are included, but are probably not necessary. Many could probably be removed after some trial and error.
  • license.txt is used by a third party camera DLL the eye tracking system relies on to function.
  • Calibrate.exe is also required, it handles calibration when an eye tracking application is started and puts the calibration results into a subfolder called EgSettings.
  • Calibration.dat and NominalCal.dat are also required files.

Eye Tracking Quirks

  • There are configuration exe's in the root Eyegaze folder that control global eye tracking settings such as calibration sensitivity and other special case tweaks. Testing so far seems to work best with basic 5-point calibration and a slightly larger tracking dot.
  • Unfortunately, the eye tracker likes some people better than others:
    • LCT claims that it works fine both with and without eyeglasses but our testing seems to disagree. Often wearing glasses can still work, but not always. If a user has trouble calibrating while wearing their glasses, they may have better luck with their glasses off, but of course then they can't see the screen as well.
    • People with especially narrow eyes also seem to have trouble calibrating and using the eye tracking system. There is a setting to try and accommodate for this in the config exe's, but we have not tested this yet.
    • People who have had corrective surgery on their eyes sometimes have a hard time as well, to the point that the system cannot reliably track them at all.
  • While the user has some freedom to move while using the system, sometimes it can lose track of them. Shaking one's head when this happens can sometimes get the system to pick them back up again, or moving slightly towards or away from the camera. It can be helpful to stay in calibration mode and have someone watch the subpictures of each eye to find a sweet spot for the user to sit.

Known Issues

  • The plugin does not always exit the Unity Editor gracefully and will sometimes stay running in the background. If you open the Unity Editor and get errors when trying to run an eye tracking project, close Unity, open the Task Manager, and end any running Unity.exe processes. Note that this error only seems to effect running from within the editor and not standalone builds.


Base Unity Project
Plugin Source (Visual Studio 2010 C++ project)