by 
Originally posted at arts.mit.edu.
“The central story I was thinking about when developing the idea of ‘embodied montage’ [for my SM Thesis in Comparative Media Studies at MIT] was the myth of Orpheus. Eurydice dies and Orpheus goes back to hell to save her. Hades says, ‘Ok, you can take her, but the only thing is you are not allowed to look back until you leave. And if you look back, she will stay in hell forever.’ At the last moment, he looks back and she stays there. So, the act of looking has a big consequence. You can create these types of moments in virtual reality very easily because it is a computational medium, and it tracks all your bodily input. If someone looks back in a VR experience, it can cause something.” To Deniz Tortum, this ancient myth speaks to something quintessential in VR projects, such as Oscar Raby’s Assent—looking signals moving, and moving equals involvement. To bear witness is no mere ocular event, but a physical and emotional act.
Tortum, an accomplished filmmaker and a VR practitioner, traces the lineage of virtual reality back to the early days of cinema: “VR is a computational medium, and it is embodied computation. It is also the first dream of cinema—the total cinema. It is not just images, and you’re not just in the theatre. You’re actually in an environment. Film theorist Andre Bazin wrote about this—that film tries to move toward being total cinema. It’s first black and white, it’s silent, and then sound is introduced, then color is introduced. Then there’s 3D, stereoscopic, and the Sensorama with smell or movement—trying to be this complete, immersive environment. VR is almost like the realization of the fantasy of cinema.”
Tortum came to MIT after studying film at Bard College. As an undergraduate, he was interested in international art house cinema and avant-garde films that were in close dialogue with the video art world. At Bard, he made experimental films and one independent fiction feature film, Zayiat, which was selected for SxSW and !F Istanbul Film Festivals. He recalls, “I was meeting independent filmmakers and everyone was struggling with what the future of cinema is. Through those times, I started not shooting film that much, and I got more interested in interactive media—web documentaries or other new media work. And that’s when I applied to MIT. Open Documentary Lab (ODL) looks at a lot of nonfiction work that uses new technologies to tell stories. My first year at Open Doc Lab, I was more focused on interactive web docs, and then I started researching virtual reality. That did several things: I started understanding the language of these new media and their affordances, and I saw their limits—or the limits they have now. That made me see the advantages of film as a medium and what it can do better than these other things. So, looking at these new media renewed my interest in film and informed my filmmaking.”
Tortum recently screened a new short film at True/False film festival called “If Only There Were Peace” (co-directed by Deniz Tortum and Carmine Grimaldi). He also is a Creative Advisor for the CMS course Hacking VR at MIT. In this interview, he speaks about his research and several of his virtual reality and documentary film projects.
Conversation with Deniz Tortum
Does your knowledge of film influence your virtual reality projects? And vice versa?
When thinking about virtual reality, I am trying to apply film history and film theory to virtual reality. Even though many people are against that, I think it is a fruitful thing. Revisiting each medium through other media is definitely helpful—especially at keeping me motivated.
Most VR practitioners or academics say, ‘VR is not a filmic medium. It is not cinema. It is completely something else. It is its own medium.’ But when they say it is not cinema, they are generally thinking of one particular strain of cinema, which is probably the mainstream Hollywood cinema. The thing to remember is that cinema is not one unified medium. There are lots of different cinema histories. I think that several of them, especially the ones that are obscured by mainstream cinema, the more experimental traditions, have much more to inform VR practice today.
That is what I am trying to figure out. I’m looking at different traditions of cinema and how they can inform VR. For many practitioners, when doing cinema, they are imagining a different future medium and not the cinema itself. For example, Hollis Frampton is trying to make these automated computer-edited films. He’s trying to do computational media with film. Even how Godard structures his films has a lot in common with the interactive documentary world. Or, for example, Michael Snow’s film, Central Region, it is basically this camera on top of a mountain connected to a machine with no humans operating it, so the machine is operating the camera and the camera is shooting continuously attached to the machine. It is a seeing entity without a human operator, creating an automated filmic space. That definitely speaks to virtual reality and 3D virtual environments.
What was the topic of your thesis for CMS?
My thesis was called “Embodied Montage.” The new image is created through 3D capture. You can capture the world as a computer graphic and process these images of the world in game engines and create a game-like interactive experience of the captured real world.
In a virtual reality system, what happens is the images and sounds react to your body’s movement and position. VR can get input from your body—it tracks your head movement or hand gestures or position. With VR you can decouple action and perception and make new pairings. The environment can respond to the body in novel ways.
In film, you have one shot and a consequent shot, and from the juxtaposition, a third meaning can appear. In virtual reality the third meaning can appear from the novel relationships between the body and the environment, between action and perception.
You received a 2016 Schnitzer Student Art Award and exhibited a VR piece about a Turkish hospital in the Wiesner Gallery as part of that awards show. Could you talk about the genesis of that project?
I’m both shooting a documentary about the hospital and making a VR piece. I was really interested in how doctors relate to the hospital and to their profession. It’s not something that we see that much; when a piece is about a hospital, it’s generally about the patients or about the institution itself. T. V. series about doctors tend to be much more sterile, and they don’t really get into the mindset of being in a hospital and being a doctor. So, I wanted to shoot a film with that in mind. I was also working with the Sensory Ethnography Lab at Harvard, taking the class with Lucien [Castaing-Taylor] there. I am continuing to work with the Film Studies Center at Harvard.
Essentially, it is an experimental ethnography—more about bodies, more about doctors. It’s observational, but in an embodied way. It is not just the camera on a tripod taking these long shots. It is much more about me as an observing subject, who is much more involved with the film. It is also not shy in showing stuff, so there are a lot of operation scenes.
And then with the VR piece, the user enters a hospital environment that isn’t populated with patients and doctors. The hospital setting evokes a lot of human drama despite the absence of people.
VR is more about spatial storytelling. Henry Jenkins, who is a founder of CMS, has a piece called “Game Design as Narrative Architecture.” He says the space you choose is the essential part of the story. That is the bone of the narrative; you build on that. Having a very charged space to build on is really an essential thing for film as well.
For my process, I do a form of 3D capture, which is laser scanning. Laser scanners are mainly used by architecture firms or engineering firms. They send millions of lasers through a place and by calculating the time the laser takes to return, it measures the distance. So it generates a point cloud of the space and you can combine the points of capture to create unified spaces.
Similar to early photography—like with daguerreotypes, where one exposure would take like 15-20 minutes—laser scanning requires a long exposure, which means you don’t capture humans unless they’re posing for like 10 or 20 minutes. So what it does is it gives you these empty places without people.
I think the hardest thing about VR is to make pieces about people, because we don’t have a good way of capturing movement and the world in real time. Film is great for capturing reality in passing. VR is almost about the absence of people; you can do effective pieces about people with VR, but you should be aware of the limitations of the technology. Of course, that will probably change within 5-10 years.
In your VR documentary about the Istanbul Pogrom, September 1955, which you presented in the Keller Gallery at MIT, you and your co-creators devised an interesting work-around for depicting people, by making the figures amorphous and ghost-like. The piece draws on the photographic archive of Maryam Sahinyan (1911-1996) and Osep Minasoglu (1929 – 2013), Armenian photographers who lived in Istanbul at the time of the attack. Could you describe this work and talk about the decision to abstract the figures?
I did this VR installation about the Istanbul Pogrom with two friends, Cagri Hakan Zaman (PhD Student, Design and Computation Group, Department of Architecture) and Nil Tuzcu (Research Fellow, Department of Urban Studies and Planning). It was funded by CAMIT.
In 1955, over two days, many shops and houses of the non-Muslim minorities in Istanbul—Greeks, Armenians, etc—were destroyed by mobs. The piece is a virtual reenactment of it. It places you in a photographer’s studio and you hear things outside the shop and experience it from the perspective of a local shop-owner.
If you put photorealistic people in the space, it breaks the essence of the whole project—that this is a lost, long gone space. These people are gone. They are not real. They were lost. So we decided to keep them as ghosts. And moment to moment, you get to see the real people from Maryam’s photographs. So what we thought was, OK, if you’re in that studio, we should be building up the story with the photographs of people who are gone. That was the idea to use some abstract representation.
If you could add something to your MIT experience, what would it be?
If there is any chance of getting a collection of Ed Pincus films here, that would be great. One thing I noticed when I came is that it seems like there is no film archive. I thought Ed Pincus’s 35mm films would be here. Making a connection to the art tradition and history within MIT more visible would be valuable. Ricky Leacock, Ed Pincus and Gloriana Davenport are great, and there are all these other great people like Michael Naimark, who worked here and is a big pioneer in VR and projection mapping. With Gloriana Davenport, he worked on Aspen Moviemap; they did the first experiments in interactive media.
The first computer game was done here at MIT. It was the incredible moment in video game history. But they were computer scientists and their work wasn’t understood as part of what would become a new medium, so the information is not in one central archive. It’s a quintessential moment in the history of a medium. Also, VR started here. The first headset was created by Ivan Sutherland when he was a graduate student at MIT. How do you bring all those archives together, or those traditions together?
