Mixed Media / Interactive 360 film
Developing an interactive, choice-driven 360 film has been attempted at the etc by Project Kalpana (2015) and Project Hindsight (2017). The starting template here is to allow for a project similar to Hindsight: on in which the interactions can happen at any time.
What is Mixed Media Interactive 360 Film?
The idea behind this form of interactive experience is to combine film editing techniques and the photo-realism of film to the unrestricted interactivity that VR can offer. Combining these two to have film react and play depending on how/when/with what the guest interacts can create a unique VR experience that hopefully can deliver emotional and engaging content.
Use of Technical Architecture
The .zip file contains a template for creating a mixed film-cg experience. Here is a short list of things to be aware of and tips.
Open the project in Unity.
- Demolition Plugin
- Currently, the film assets are displayed on the sphere using the Demolition Plugin. This allows for high-quality video to be displayed in 360 without having awful framerate issues.
- Demolition uses videos in HAP format to take advantage of the GPU. Make sure the videos that you want to use are in the StreamingAssets folder and are in a HAP format.
- The SceneManager is what manages what clip is playing and what frame that clip is on. Each clip of video is used by a Demolition Media Container and then paired with an enumerator known as a Scene. Information about Scenes are in the SceneInfo class. This contains a dictionary about scene progression, loop information, and interruption information.
- The SceneManager is set up to allow for videos marked as loops to loop for a defined number of seconds. This means that footage can loop if you are waiting on guest input.
- There is also functionality for interruptions. Interruptions are specially marked scenes that play out of order and return to the last scene that was playing when they finish. This can be modified in the SceneManager as well.
- The VideoEventsManager is what allows for responsive behavior. There are currently 3 different types of events the manager deals with.
- Video Events - These events are tied to a certain frame in a scene. - End Scene Events - These events trigger at the end of a certain scene. - End Loop Event - These events happen at the end of certain scenes that are marked as a loop. These are triggered when the loop scene ends, not when the video clip ends.