(Originally published on Blogcritics.org)
Porter Stowell, from IBM, discussed combining Watson with VR
South by Southwest (SXSW,) the annual Austin, Texas, music, movie and technology conference included a track on the new technology of virtual reality (VR) and augmented reality (AR). Although some aspects of VR/AR go back to the 1980’s, it’s “new” because the hardware has become affordable for consumers and game engines used to create VR have become more powerful. Now there is a market attracting a wide range of companies from start-ups to ancient tech giants like IBM and Microsoft.
The question that dominated many of the seminars at the conference was “How do you tell a story using this new technology?”
Medical Minecraft
IBM’s solution is to turn the story over to its cognitive software: Watson. This was the computer software that won at Jeopardy. In his talk “Games Plus Cognitive Equals Game Changer!,” Porter Stowell, Head Of Strategy and Business Development for IBM Serious Games and Gamification, gave examples of how the power of Watson was helping people from high school to secret government agencies.
In a high school in Austin, IBM has created a Medical Minecraft game using the same database backend as doctors use to diagnose patients. “When you say to high school students, ‘Let’s study science’,” Stowell said, “you often get an eye roll. But if you say, ‘Let’s play a game,’ you get an enthusiastic ‘Yes!’.”
IBM’s efforts go beyond the classroom. Companies and agencies can use Watson and VR to train security teams to battle hacking attempts. Watson can learn the rules that organization has for its security and use them in the game.
On a national level, IBM created games to train interrogators. This works because Watson is able to, for instance, dump a twitter feed to get a feel for a suspects personality and because it remembers context.
If you ask Watson, “How many moons does Jupiter have?” and then ask “What about Neptune?” it will know you are referring to the planet, not the Greek sea god.
Look into Henry’s Eyes
At the session “Oculus Story Studio University”, SouthBy attendees learned from Oculus writers and producers the challenges in telling stories for this new medium. They referred to it as “Immersive Cinema.” Ramiro Lopez Dau, Director and Animation Supervisor at Oculus VR, discussed lessons from his story Henry.
‘Henry,’ the story of a hedgehog created in VR, presented challenges to the filmmaker
Henry is a hedgehog that is alone and sad on his birthday. No one wants to hug a hedgehog.
“Henry is different than a movie,” Dau said. “Here you are sharing a physical space with a character. We started like a regular movie with storyboards. Then we realized we had to change things and not think like a set designer, but like an architect. The house has different levels; it’s not just a backdrop.”
He continued, “We discovered you can’t just push stuff on people right away. You have to give them time to explore. To feel comfortable. As the storyteller, you have to let go.”
Dau said that there were unknowns with the acceptance of this medium. “Henry looks into your eyes several times. But, if he acknowledges your presence, then he’s not really alone and that undercuts the story. We thought of it as like breaking the fourth wall.
“When we showed it to people in the industry they deconstructed it and objected,” he said, “but, when we showed it to regular people they didn’t question it. Henry shares those moments of loneliness with you. Logically, this shouldn’t work, but it’s a new medium, so we will show it to audiences and wait for a reaction.”
Transformational Experiences
The rules for VR filmmaking differ from those for traditional film
At the “VR, AR and Digital Storytelling” session, participants believed that now that everyone was aware of the technology, they had better start telling good stories or miss an opportunity.
According to panelist Tony Christopher, President of Landmark Entertainment Group, this is a sea-change event.
“We are no longer in the game business,” he said. “This is not a game. It is not a movie. It is a new medium. How often does that come along? When people first started making movies, the movies looked like theater.”
Christopher explained that just as audiences had to become used to the conceits of movies – close-ups, montages, split-screens — consumers and filmmakers now must go through a similar learning and discovery curve.
Lorraine Bardeen, Director of Studio Business, Strategy, Operations at Microsoft, said she was fascinated to see how people were bringing VR/AR apps into their personal space.
“Using Microsoft Hololens,” she said, “you can build a Minecraft world in your living room. I think it’s fun herding virtual sheep around my living room, but there are many serious apps as well.”
Lorraine Bardeen from Microsoft shared her adventures with Hololens
Bardeen explained that anatomy students could examine a living, virtual body. “This will explode their ability to learn. They will be able to see what they will be experiencing when they are doctors.”
Bardeen said that storytelling was not just for professionals. “Actiongram, a story telling application, is the first wave of tools that will let people put holograms in their world and then record them,” she explained. “We give you a wide variety of characters and props and you use them to tell your story. Our app focuses on funny stories for online but there are other stories you can tell.”
“Some people use it in beautiful ways,” she said. She laughed and then admitted, “But my daughter just takes pictures of me, gives me googly eyes, and then makes me explode.”
You can see Actiongram in action in the video below.