Development for Technology in Curricula

A little over a year ago, our expertcenter started working alongside other parts of HKU on training teachers to incorporate relevant (new) technology as form or content of curriculum. The pilot for this program was run with a group of staff form the School of Media.

My role in this process was to advise on choice of technology, implementation feasibility, and assist with the distribution of development requests.

One topic that originated with multiple participants was the question of interaction in a 360 degree media environment (photo/video). After some research on current tools / processes, we concluded it was better to adapt an existing tool for our specific use-cases, because much of what was out there was too limited in scope or inconsistent in process-of-use for our wishes.

A funny detail in this case is that if you google “interactive360” today, a lot of services and tools are now available that did not exist 12 months ago when we were looking into this. It seems a lot of people sought to step into that particular void!

Selecting from existing or building yourself

The question to develop something yourself is one that comes up a lot, and in my opinion is usually answered too quickly (whether the answer is yes or no). Some people assume what you buy will always suit you better, and others will assume the opposite. Neither is always true, so it helps to question what you need, and what you need it for.

In our specific case we wanted something for a broad scope of possible concepts (interactive photo collages, 360 video narratives, conceptual cross-overs), which meant it needed to be a flexible authoring environment. This also precluded a lot of free “off-the-shelf” online tools that did very specific things (360 photo/video point-and-click-through).

What helped us decide was that we had chosen to focus our development for tools on Unity, a reasonably easy to develop for platform that incorporates many different workflows, and that Unity themselves had built a similar type of tool, called Interactive360.

Interactive360

The good part of having an existing tool like Unity’s Interactive360 is that a lot of the technical hurdles are already solved. The biggest problem is that the built solution might not be exactly what you want, which turned out to be the case for us. The use-case that they had built the tool around was “laboratory”, narrow in application scope, and almost comically “game-like” in narrative and workflow. The project even contained a code hub called the “GameManager”, which means absolutely nothing to non-technical Media staff, and probably just confuses them more than it helps.

What they’d built it for was a simple previewing situation in which every scene could transition to every other scene (literally a tourist situation). What I wanted allow staff and students to build is (for example):

  • A 360 interactive “Murder on the Orient Express”, where a number of viewpoints are synchronously played from a re-enacted recording (12 viewpoints = 12 videos of each viewpoint over the same timeline), and then allow players to hop from one viewpoint to the other when they come into contact with these characters.
  • Contextual information for objects when you look at them (spoken audio recordings).
  • Or perhaps a single long-form video, and jumping to different points in time as a time-travelling narrative.

Unity’s tool was not suited for those kinds of experiences.

What we needed

To get from a mostly functional but confusing/conceptually inadequate Interactive360 project to something more user friendly / applicable, we recoded the workflow to work entirely form an “in-scene” editing situation. Once an environment (photo/video/etc.) was setup, hotspots could be created (you look at them, and they serve as triggers), and from these hotspots all relevant interactions could be controlled (scene transitions, audio playback, animation, etc). This made much more sense than describing all of the scene-flow information up front (which could become very complex very quickly!), and confronted you only with what was relevant in the context you were in (the selected hotspot in the current environment).

Testing, 1..2..3…

As a first iteration, we built a system that only supported environment transitions, and had separate parts that you edited for specific purposes. These didn’t exactly match what users were trying to do (e.g. moving from one scene to another and restarting meant setting the target environment as “not-looping”), which meant it was necessary to wrap those things into a consistent single-view system.

To accomplish this (and a few other quality-of-life improvements), we added a preview window. Unity allows the creation of entirely custom windows that do whatever you want, and in this case it renders a view from the camera as if we were the end-user watching the experience. This makes sure that “what you see is what you get” (a hallmark of authoring usability).

On top of this, we display context-dependent information boxes containing all the data about hotspots you have selected, and if you select an in-scene video player it allows you to jump to different frames, hit play/pause, much as you would expect in video-centric environments (although not quite as smoothly).

Hotspots and scrubbing
Hotspots and scrubbing in the Preview Window

Work-in-progress

The tool as it currently exists can be found on our github. To make it easier to use, it includes a guide, currently in Dutch. Using the tool requires some basic Unity knowledge (the more the better), and links are included in the guide to relevant manual pages to read up on Unity.

In its current state the tool is a prototype. Iterations are not consistently planned, but are dependent on use of the tool. If the tool finds more of an audience, we can reserve more time to make sure it matches the necessary workflows. And as always, if something pre-made comes along that better (or more quickly) serves the desired purpose, that tends to be the safer bet.

If you do end up trying it out, make sure to add desired additions / changes to the wishlist at the end of the guide!