yamammy: A short VR documentary.

 

yamammy is a virtual reality documentary that explores how displaced refugees navigate a world in which they possess disparate overlapping ethnic, religious, and regional identities.

In this project, a former refugee from Sierra Leone Yamammy describes her feelings about some major life transitions – moving from Sierra Leone to Guinea to New York to a small town in Idaho – and how her sense of home and identity has evolved. With this doc, we wanted to create a fragmentary digital portrait of Yamammy using images and stories she describes from her memory.

In class, we presented a rough cut of the 360 video version of the documentary (still a work in progress, so publishing date TBD).

For the ITP Winter Show, we built the VR version of the documentary, which allows audience members to some of the images and stories that Yamammy describes. See the teaser:

We wanted to give the user some autonomy but not too much autonomy. There is some movement but we wanted the user to focus on the audio. Ultimately we wanted Yamammy’s voice to dictate the pace and the rhythm of entire experience.

The concept.

When Ruta and I first discussed this project, we both agreed that we wanted the story itself to determine the technology that we would use. We were both interested in exploring the idea of locative memory: memories that are bound to location or spatial dimensions.

It wasn’t until I had an initial conversation with Yamammy that the documentary began to take shape. We realized that many of the experiences she described were disjointed. Often she would remember tiny specific details while glossing over other parts of her life. Memory itself functions in this way: it’s fragmentary, it’s unreliable, and we tend to remember the stories that match patterns.

Photogrammetry as an aesthetic seems to capture the feeling we wanted our documentary to evoke. Tiny details come into focus while other details are distorted, twisted. It’s an imperfect process, just like the human recall.

The audio.

I initially did a pre-interview with Yamammy to gauge how she might feel about telling some of her stories. It went well and so Ruta and I moved forward with the full interview, which we conducted over Skype. From the two hours of audio, we then combed through the interview and edited it down to a handful of distinctive stories.

The visuals.

We started finding footage online based on Yamammy’s stories. We wanted to capture some of the landscapes she described in an abstract way and so drone footage became a useful creative tool for us.

ffmpeg

We then generated individual frames from the footage and pulled those images into Photoscan, where we created points, then point clouds, then a mesh. After exporting the model from Photoscan, we cleaned it up and smoothed it out in Meshmixer, exporting it as an .obj and a .jpg texture.

untitled-7

We then brought the objects into Unity, where we arranged them into various “scenes” for each audio story we had edited.

light

For the 360 video, we used a Unity plugin called VR Panorama, which allows you to animate a camera that flies through a scene. We edited those videos together with the audio in Adobe Premiere to create the 360 video we presented in class.

For the VR experience, we worked with the Samsung GearVR. We went back into Unity and animated the camera to inch slowly through each scene. We added the audio story to each scene. We also wrote a C++ script that would trigger the next scene when the previous scene had ended.

The feedback.

We showcased the VR experience in December 2016 at the ITP Winter Show. It functioned as a user testing of sorts, since Ruta and I still felt as if the documentary was in a rough place. We received some really good feedback about the experience which included:

  • Tightening up the stories to reduce overall time length.
  • Thinking about replacing some of the visuals – the visuals that worked best were the landscapes.
  • Emphasize the collage effect/aesthetic.
  • Rethink using the Samsung GearVR, as the user experience is not the best (the trackpad is too sensitive, the headset doesn’t fit everyone’s face).
  • Adding music to emphasize certain emotions.

We’re planning to continue working on the VR film and hopefully will submit it to some film festivals this spring.

Leave a Reply

Your email address will not be published. Required fields are marked *