As we look back at the evolution of visual media we see a collective experience that was shared in the dark, be that in a theater or at movies. We then migrated our primary interface to the TV. With the advent of the Internet our screens became pervasive and ever present. We now sit at the cusp of the next evolution in our media user experience (UX). This change will be the biggest one yet.
Not only are we talking about a change in the consumption of media but the media itself being aware of the environment. By augmenting not only our reality but by utilizing visual recognition these media experiences are much more than anything we have ever experienced. The entire construct of how we can tell and weave our story worlds is changing and I think it’s time to bring to front the creative conversation about how best to utilize these new tools in a story tellers tool belt.
These new tools are upon us. The promise of VR has been in the collective conscious for more than a decade and we are now entering that virtual reality. A story in 360 degrees. Our imagination is no longer limited to a flat square, be it at 4K/8K or beyond it’s still a frame. We now play out our narrative as an environment. A tapestry that was once left to the imagination can now be crafted as a fully immersive world where can evoke our chronicle in a deeper, richer and more realistic fashion.
To me VR is a great tool to correlate that theater experience, alone together in the dark, a personal closed experience. Even with interacting in virtual space physically you are inside the machine, if you will. Augmented Reality to me seems to be a very interesting and more communal and visceral experience. When we bring the existing environment into the narrative it creates an amazing opportunity to integrate into the fabric of our real life. If you look back at the research Latitude did on The Future of Storytelling you can see this clearly is something the audience yearns for.
Think about the other massive change, our content can now see us. Let that sink in… Ok so now my protagonist has to not only concern himself/herself with the other characters in the story, but now is breaking the fourth wall to interact with the audience. How does this change our structures for story? How does the Hero’s Journey change? How will we change the patterned narratives and themes that pervade our story structures? How does it effect core plots like Rages to Riches, Voyage and Return or Overcoming the Monster? If we become the monster that must be overcome how does that effect the flow and dynamic of our tale.
So let’s take a look at the technology that allows us this new creative freedom to build our 21st century legends. We are very close to the consumerization of a number of systems that rely on glasses to display or project 3D elements into our field of view. As well these devices will also know our orientations and most will have mapped our environment.
We have a number of players in this space and one of the ones I am very interested in is Magic Leap. To me they have a very distinctive advantage in that they are using projected packets of light directly on to the eye to seamlessly integrate the media into our natural field of view. Because of my exposure to visual recognition at Microsoft Research Labs back in the 90’s I have been thinking about this for quite awhile. To me the power is in a system that can track the eye as well as the environment in which the field of view is examining. This allows for the seamless integration that is portrayed by the example video’s that Magic Leap is putting out.
This visual awareness in the context of a mapped environment matched with the stereoscopic projection of 3D elements is a game changer. The capacity of this technology as a story telling tool has vast implications in film, TV, games, education, medicine, advertising and just about every facet of modern life.
Since 2002 we have envisioned a Minority Report styled interface to our systems and media.
We dreamed of our Holodeck and how we would touch our data and entertainment in a truly interactive and immersive fashion. Well we now stand poised to enter that reality and live in a world where yet again technology has caught up with our imagination. Fantasy is fact and now we get to ponder the questions of how to make this real technology a factor in our media creation. How do we deal with color correcting in a 360 environment? How do we deal with light occlusions and simultaneous interaction with projected voxels? How do we tackle the narrative and storytelling changes put forth above? How do we as artists and technologists craft the magic that is media in this world where, even if not real, the things we create are visually a life like fairytale?
Boy this is going to be fun!