Project proposal redux.

Terrapattern, Golan Levin

For this week’s assignment, we were to reframe or revisit our project idea through a scientific lens. Since computer vision — characterized by image analysis, recognition, and interpretation — is itself considered a scientific discipline, I struggled to find a new scientific framework through which to re-articulate my project.

Because my project is so deeply rooted in computer vision and optics, I’m interested in exploring the idea of “algorithmic gaze” as the means by which computers categorize and label bodies according to specific (and flawed) modalities of power.

Donna Haraway’s concept of the “scientific gaze” has very much influenced my research. In her paper “Situated Knowledges: The Science Question in Feminism and Privilege of Partial Perspective“, Haraway tears apart traditional ideas of scientific objectivity, including the idea of the subject as a passive, single point of empirical knowledge and the scientific gaze as objective observer. Instead, she advocates for situated knowledge, in which subjects are recognized as complex and the scientific gaze is dissolved into a network of imperfect/contested observations. In this new framework, objects and observers are far from passive, exercising control over the scientific process.

Haraway relies on the metaphor of vision, the all-seeing eye of Western science. She describes the scientific gaze as a kind of “god trick,” a move that positions science as the omniscient observer. The metaphor of optics, vision, and gaze will be central to the development of my project. I’m interested in exploring how the “algorithmic gaze” mediates and shapes the information we receive.

Sandro Botticelli (Florentine, 1446 - 1510 ), Portrait of a Youth, c. 1482/1485, tempera on poplar panel, Andrew W. Mellon Collection 1937.1.19
Sandro Botticelli (Florentine, 1446 – 1510 ), Portrait of a Youth, c. 1482/1485, tempera on poplar panel, Andrew W. Mellon Collection 1937.1.19

My first test was using ConvNetJS, a JS library built by Andrei Karpathy that uses neural networks to paint based on an image as input. I used a detail from the painting above and ran it through the neural network. Here’s an example of the process.
screen-shot-2016-09-29-at-2-53-35-pm screen-shot-2016-09-29-at-2-54-23-pmscreen-shot-2016-09-29-at-3-00-11-pm screen-shot-2016-09-29-at-2-54-29-pmscreen-shot-2016-09-29-at-3-03-10-pm

Midterm project proposal: Biometrics.

Problem framework.

For my midterm project, I’d like to address the ethics and implications of widespread biometric data collection.

Biometric identifiers are defined as measurable, distinctive characteristics that are used to label or describe individuals. They’re commonly used by governments and private organizations to verify the identity of an individual or group of individuals, including groups that are under surveillance. Physiological characteristics include fingerprints, DNA, face recognition, retina scans, palm veins, hand geometry, and iris recognition. Behavioral identifiers measure behavioral patterns like voice and gait.

Here’s the breakdown of identification accuracy based on biometric input:

screen-shot-2016-09-27-at-5-33-40-pm

The earliest record of fingerprinting cataloguing dates back to 1891. Biometrics arguably originated with “identificatory systems of criminal activity” as part of a larger system to categorize and label criminal populations. “The biometric system is the absolute political weapon of our era” and a form of “soft control,” writes Nitzan Lebovic. Under the post-9/11 expansion of the Patriot Act, biometric systems have expanded from the state to the private market and blurred the lines between public and private control.

While biometric data is seen as being more accurate and therefore more reliable as a way to identify an individual, it is also not replaceable. If your private password was somehow compromised, for instance, you could simply change your password. On the other hand, you can’t replace your fingerprint or change other physical characteristics.

Italian theorist Giorgio Agamben experienced the implications of “bio-political tattooing” firsthand in 2004 when he was told that in order to obtain a U.S. visa to teach a course at New York University he would have to submit himself to fingerprinting procedures. In a piece published in Le Monde, Agamben explains why he refused to comply, arguing that the electronic filing of finger and retina prints required by the U.S. government are ways in which the state registers and identifies naked life. According to Agamben, biometric data collection operates as a form of disciplinary power.

Audience.

This issue affects potentially everyone so our audience is very broad. Biometrics data is most often collected about populations that are already vulnerable, including criminals, the poor, and immigrants. Corporations put a monetary value on biometric data, and yet individuals don’t think about data collection as an intrusion.

The goal of this project is to foster an awareness of the implications and ethics of biometric data collection.

Ideas for the project and user journey.

Concept #1: A physical installation that gives the user personalized information based on a biometric input.

Concept #2: A speculative VR experience with advertisements tailored to user’s biometric data.

Concept #3: A kit of wearable objects aimed at masking and altering user’s personal biometric identity.


http://motherboard.vice.com/read/i-replaced-my-fingerprints-with-prosthetics-to-avoid-surveillance

Concept #4: Collect (non-identity-compromising) biometric data from various participants and sell data on eBay in order to gauge the monetary value of the data.


http://thecreatorsproject.vice.com/blog/this-artist-turned-herself-into-a-corporation-to-sell-her-data

 

Redesign of a narrative experience: Mrs Dalloway in 360.

screen-shot-2016-09-20-at-5-32-44-pm

She would not say of any one in the world that they were this or were that. She felt very young; at the same time unspeakably aged. She sliced like a knife through everything; at the same time was outside, looking on. She had a perpetual sense, as she watched the taxi cabs, of being out, far out to the sea and alone; she always had the feeling that it was very, very dangerous to live even one day.
– Virginia Woolf, Mrs. Dalloway

For this week’s assignment, we were asked to redesign a narrative experience according to the agile human-centric design principles we discussed last week.

For my source material, I drew from the themes and text of Virginia Woolf’s Mrs Dalloway, a 1925 novel written in a stream of consciousness literary style that sketches the portrait of life of one woman, Clarissa Dalloway, during the course of one single day.

In the first chapter of the book, Clarissa walks the streets around London running errands in preparation for a party she is throwing that night. When I reread the book, I was struck by the ways in which the novel sharpens our attention to details of time and space, especially the specificity of London during Clarissa’s walks. Time is a significant theme in the novel, with clocks ringing the hour and signs of aging and death made hypervisible in the text. So much of the narration in the novel occurs inside the head of the protagonist, with special attention paid to her surroundings.

With this project, I wanted to explore creating a film that employs this stream of consciousness narrative style while physically putting you in the shoes of the protagonist. I chose to reimagine Mrs Dalloway as an immersive VR/360 experience in order to explore this narrative style not only in text, but also in film.

The idea behind the project was to film myself walking in New York using 360 video, paired with a voice over narration of the opening chapter of the novel. I made slight changes to the text in order to accommodate the sharp departure in setting (from 1925 London to 2016 New York). Much of the narration in the novel is observational — Clarissa sees a woman in a taxi cab, she arrives at the park, she looks in shop windows — and I wanted to replicate those moments in the film as much as possible.

Check out the initial prototype of my idea.

YouTube:

 

My audience for this project could be anyone, really. Because it’s a 360 video, the user has full control over what he or she is looking at during the film. Just like London, New York is replete with observational details; I wanted the audience to experience that same sensory overload in my project.

Vimeo:

How the internet works: Understanding networks.

For this week’s assignment, I performed a traceroute on three sites I visit regularly from all the places I regularly connect to the internet. A traceroute is a computer network diagnostic tool that displays the route that packets take across an IP network. The route is recorded as the packets are received from each successive host (remote node).

To start, I examined the route the packets take to get the servers that host bit.ly, a website that is popular for generating short urls. I had noticed a few years ago that .ly is the internet country code domain for Libya, which seemed unusual to me. When I investigated further, I read this article that lays out the implications of adopting a domain name that’s associated with a country with an authoritarian or unstable government.

I traced the packets’ route, which revealed that the packets jumped from U.S. servers to a Swiss server in Zurich, and back to the U.S. See the journey here:

Project Development Studio: Project proposal.

Project proposal.

I intend to use this class to explore generative text as a new poetic form, culminating in the production of some kind of physical or digital artifact.

Over the coming semester, I will conduct a series of text-based experiments using deep learning methods such as Recurrent Neural Networks (RNNs) for sequence learning and Convolution Neural Networks (CNNs) to classify images and text. I’ll also use Python (w/ Flask), Javascript (w/ Node), and Natural Language Processing (NLP) libraries in both of those programming languages. The goal behind these experiments is to teach myself different ways of training a computer program on text to generate something new.

I’m still not sure what form the final artifact will take, whether it’s a physical book, an installation, an interactive web-based tool, a chatbot, a mobile app, or otherwise. My hope is that the form will eventually emerge through my experimentation.

Some major questions I still have about this work deal with the audience response. What can I build that will elicit an emotional response? Will people understand the intent of this project? How will they connect with it if they aren’t writers/readers/theorists?

Here’s the project map I sketched out during our class activity:


img_6271img_6272

Next steps.

Since I’m still unfamiliar with some of the tools I’d like to use, for the first few weeks I intend to teach myself the basics of deep learning. I plan on using resources from Gene Kogan’s course Machine Learning for Artists, Patrick Hebron’s course Learning Machines, and Andrei Karpathy’s amazing work on RNN. I’m going to build a week-to-week schedule to lend some structure to my experimentation. I’m taking another Javascript-based generative text class right now, so my experiments might align with that class as well.

Resources & inspiration (an ongoing list I will update).

 

Other ways of telling: Andrei Tarkovsky.

Listen: you are not yourself, you are crowds of others, you are as leaky a vessel as was ever made, you have spent vast amounts of your life as someone else, as people who died long ago, as people who never lived, as strangers you never met. The usual I we are given has…none of the porousness of our every waking moment, the loose threads, the strange dreams, the forgettings and misrememberings, the portions of a life lived through others’ stories, the incoherence and inconsistency, the pantheon of dei ex machina and the companionability of ghosts. There are other ways of telling.

― Rebecca Solnit, The Faraway Nearby

One of the goals of VR documentaries, suggested by the filmmakers behind Collisions and Clouds over Sidra, is to give the audience a sense of ‘presence.’ Immersive experiences in VR allow the participant to feel that he or she is physically in the same location as the camera. The technical constraints of VR filmmaking demand slow, deliberate camera movements. Many VR films have a structured narrative while still giving the audience the agency to decide what they will look at during the film. To paraphrase one reviewer of Clouds over Sidra, what moviemaker in the past would include 20 seconds of ceiling shots, looking up at the top of the tent?

I include these observations about VR film because I’d like to highlight the ways in which Russian filmmaker Andrei Tarkovsky uses similar techniques in many of his films to produce an immersive, experiential cinema experience.

During his career, Tarkovsky directed only seven feature films, including Solaris, Stalker, Andrei Rublev, and Mirror during the period 1962-1986. He was a pioneer in the field of cinema, experimenting with new narrative techniques and theories. Many of Tarkovsky’s films are characterized by extremely long takes, slow camera pans, and very few cuts. He developed a theory of cinema called “sculpting in time,” in which he explored how film can twist and alter the audience’s experience of time. Unedited movie footage and lengthly sequences were used to heighten that feeling of time passing.

I watched Solaris for the first time in college and I remembered being shocked by a long, drawn-out driving sequence within the first ten minutes of the film. The scene is nearly five minutes long.

Tarkovsky writes: “If the regular length of a shot is increased, one becomes bored, but if you keep on making it longer, it piques your interest, and if you make it even longer, a new quality emerges, a special intensity of attention.”

Tarkovsky also employed common motifs of running water, clouds, and reflections in his films. Many understood his preoccupation with reflective surfaces to mirror his own interest in self-reflection and introspection. Of Tarkovsky, Ingmar Bergman said: “Tarkovsky for me is the greatest (director), the one who invented a new language, true to the nature of film, as it captures life as a reflection, life as a dream.”

On a personal level, I’ve learned from Tarkovsky that compelling films do not need to have a strict narrative, nor follow time limits or other cinematic constraints. Tarkovsky’s films were so powerful because they pushed the audience into a state of heightened attention.

Crafters of immersive/experiential films often need to make similar decisions about timing, camera movement, and narrative in order to tell the most compelling story possible. The VR film Collisions, for instance, mixed beautiful, wide landscapes with meandering narration in very lengthy shots. With my work in this class, I’m interested in exploring the kinds of experiential, non-linear narrative that Tarkovsky’s films often embody.

Here’s the presentation I gave in class:

Finnegans Wake-ify: An experiment in generative text.

For this week’s assignment in Programming A to Z, I built a simple web application that allows the user to input a paragraph of text and a computer program outputs a text that sounds like the stream-of-consciouness writing style characteristic of James Joyce’s Finnegans Wake.

Test out the generator here.

Example output (from an email I’d written):

You scraggy riverrun kidding. I completely agree with you – Eve reason shore war: recirculation interesting to me war: swerve contradictions. commodius shore bay Fantasy, for commodius commodius song “Runaway” catches you off guard vicus it war: recirculation bay Howth Castle to Eve rest Environs commodius album. recirculation every “Runaway,” Howth Castle there scraggy 10 iterations Environs passen-core am isthmus Sir Tristram I scraggy he’s captured isthmus picture Environs Europe kind Environs self-aggrandizement to side penisolate artists become victim. The real question at stake Europe war: to wielderfight degree he war: self aware. vicus everything depends on penisolate doesn’t it? We can’t tolerate blind egoism. It has to be in some way self-referential or winking or ironic.

The web app was built in ReactJS + Webpack + Node.js and you can find the full code repo here.

To do some of the text manipulation, I used nlp_compromise, a node package that allows you to use Natural Language Processing (NLP) tools in Javascript. I was disappointed, however by how limited the tool was. Next week I plan to explore using the RiTA software toolkit with NLP.

Subreddits talking about a particular subject in the last week.

Screen Shot 2016-05-09 at 4.34.25 PM

I attended a session at the Theorizing the Web conference last week about text-based online communities such as Reddit and 4Chan. Text-based communities are ones in which there is a reciprocal relationship between participants and a text: Community members both shape and are shaped by the words that are exchanged in the online conversation.

I’m interested in exploring the kind of language that is being used in subreddit communities. Using an API endpoint I found here and d3.js, I built a simple interactive graph that allows you to see which subreddits have been talking about a particular word in the last week.

See the full visualization here.

You can find the full repository here at GitHub. Some d3.js code below:

Internal Security Zones: Generative instructions for prison design & maintenance.

facility4692

As of 2016, the U.S. criminal justice system holds more than 2.3 million people in 1,719 state prisons, 102 federal prisons, 942 juvenile correctional facilities, 3,283 local jails, and 79 Indian Country jails as well as in military prisons, immigration detention facilities, civil commitment centers, and prisons in U.S. territories (source). The American prison population has more than quadrupled over the past 25 years, an increase largely driven by heavier penalties for non-violent offenses (source).

Michel Foucault reminds us that power is not static, nor does it emanate from a center of origin. Rather, power exists in an enmeshed network and is wielded by people or groups by way of “episodic” or “sovereign” acts of domination (source). Power is dispersed and pervasive rather than concentrated, embodied, and enacted. Confirming Foucault’s diagnosis of “modern societies of control” (a term used by Gilles Deleuze), Giorgio Agamben argues that biopower operates in physical spaces known as “zones of exception,” physical spaces in which disciplinary power is exercised (source).

In many cases, the architecture of a prison – the panoptical design, the single, impenetrable cells, the isolation and surveillance – causes prisoners to internalize discipline. I’m interested in the material form of prisons. Who designs them? What do the physical spaces look like? What are the material exigencies of daily life?

60

98

For my project, I decided to generate a series of instructions for constructing and maintaining a prison environment. I used the U.S. Department of Justice’s Jail Design Guide as primary source text. I also included a list of business advice and aphorisms as well as Jorge Luis Borges’ “Library of Babel.”

Screen Shot 2016-05-03 at 12.48.11 AM

I created a template that included (1) a title; (2) a chapter subtitle; (3) a set of instructions; (4) a list of material needs; (5) key questions; and (5) a random photo from the handbook. I then used Markov chains to generate some of the text.

The result sounds something like design instructions for a dystopian prison of the future. Here were some generative texts:

Screen Shot 2016-05-06 at 11.05.53 AM Screen Shot 2016-05-06 at 11.03.23 AM Screen Shot 2016-05-06 at 11.07.02 AM

I’ve built a Flask application that allows you to generate a new text and photo when you click the button “Generate another.” Right now it’s still locally hosted but I plan to push it online soon.

See my full GitHub repository here. Below is a snippet of the Python program I wrote:

Life expectancy of people in U.S. states – an interactive map

Screen Shot 2016-04-28 at 12.27.32 PM

Last week, I found data from The Health Inequality Project detailing the average life expectancy for people who live in various states in America, male and female. I took the information from that dataset and plotted it into an interactive map of the United States.

Check out the interactive map here. 

I’m still troubleshooting some problems with the tooltip, which allows users to hover over each state and display the data from that particular state. I seem to be having trouble pulling the data from the corresponding CSV.

You can see my full GitHub repository here.

Here is the full index.js code: