The photobooth: An interactive film using p5.dom.

In class on Thursday, we were introduced to the powerful dom library in p5. According to the p5 reference article about p5.dom, the library allows you to interact with HTML5 objects, including video, audio, text, and your webcam.

I was immediately interested in trying a first pass at making an interactive film in which the user could click a button to jump to another film. I knew that I wanted to make some kind of super cut using p5.dom.

Here’s an unfinished, unpolished version of my sketch. I’m still working on it.

I was inspired by the Bob Dylan music video for “Like a Rolling Stone” in which users could “channel surf” as different individuals sing the lyrics to his song. I also was thinking a lot about video artist Christian Marclay’s art installation The Clock, a 24-hour montage of hundreds of film clips that make real-time references to the time of day. The video clips are all tied by one thing: The presence of a clock and/or time. The result is an eerie, fragmentary portrait of what one day looks like in the movies.

clock2_2353636b

I also wanted to access the webcam in some way. I’m taking my cues from Paul Ford’s insanely well-written and lengthly Bloomberg piece “What is Code,” which accesses your webcam and automatically prints a PDF certificate of completion with your picture when you have completed the 38,000-word article.

With that in mind, I wanted to combine both ideas and build a photobooth. You can switch between disparate clips of characters using a traditional photo booth in different movies by clicking the button “span time.” You can press “play” or “pause” to stop the film:

movieButton = createButton(‘play’);
movieButton.position(700, 500);
movieButton.mousePressed(toggleVid);

 

 

 

Pcomp midterm proposal: The water harp.

f5ce8a5716220b4adbae524670be1ac1Photograph by Eric Rose.

In keeping with the general tenor of my physical computation projects, I will continue to look at creative ways to provoke interactions with water.

I want a lot of my future projects to be an exploration of cymatics – a subset of modal vibrational phenomena in which a surface is vibrated and different patterns emerge in some kind of medium (paste, liquid, water, etc). Cymatics is essentially a process by which soundwaves are made visible. I like the idea of measuring a person’s heart rate and then visualizing that vibration pattern in a liquid, for instance.

The more I thought about this midterm project, though, the more I was struck by the delightful feeling of running one’s fingers through a steady stream of water. I want to build the entire interaction around that tactile experience.

So here’s my proposal: I plan to build a water harp. This is the initial sketch of the project:

IMG_2389

The harp will consist of a rounded plexiglass board that water flows over, creating a waterfall effect. The water will hit a series of 8 sensors (either moisture sensors, photosensors, or another conductive material). There will be a water pump that pumps up the water and brings it back to the top.

Each sensor will be paired with a sound of a different frequency that will play from the computer using p5.js. I’m still trying to decide what kind of sound will be best suited to this project. It could be a series of different noises triggered by each sensor (such as rainfall, thunder, rivers, etc). I was also thinking a lot about using human voices singing at different pitches that would then harmonize with each other.

When the participant runs his/her hand through the waterfall, it will create gaps in the water, triggering different sensors. Overall, I want the experience to be as tactile and delightful as possible.

 

Haikus with Donald Trump.

Screen Shot 2015-10-14 at 2.07.44 AM

I have a confession to make: I couldn’t figure out how to get this project up and running with a potentiometer in time for class on Wednesday. Keep in mind that the process was littered with tiny successes and failures and here’s just a bit of what I learned.

The assignment this week was to review the labs we did in class and figure out a creative way to get the Arduino to communicate with p5.js using serial communication. For instance, we could push a button and display text, twist a potentiometer and create an animation, touch a pressure sensor and play music, etc.

I’ve been talking a lot lately with friends at ITP about generative text and computational poetry. I loved the idea of creating some kind of random haiku generator based on the transcripts of Donald Trump’s speeches.

I figured out pretty quickly that I would need to learn a lot more about Python in order to write a program that would analyze large bodies of text. I did learn, though, that Python has a function that allows you to count the number of syllables in each word, which will come in handy when I actually decide to build that version of the project.

So I decided to hack the project anyway. I searched online and realized that a few other people have been paying attention to Donald Trump’s campaign speeches and written down examples of unintentional haikus he’d said in public.

I pulled 15 haikus from a Washington Post article and loaded them into a spreadsheet. I divided each line of the haiku into a different column in the spreadsheet and called each column individually as an array, p. That way, whenever you clicked your mouse it would randomly cycle through the haikus. I was able to display a haiku by asking for line1[p], line2[p], and line3[p] whenever the mouse was clicked.

Success! I created a version of the haiku generator in p5 before adding the physical input we’d learned about in the labs. Check out the Donald Trump haiku generator here.

Screen Shot 2015-10-14 at 2.10.04 AM

Next, I added in some code to enable the serial communication between the Arduino (already loaded with some simple commands) and the computer’s serial port.

Screen Shot 2015-10-14 at 2.12.48 AM

I mapped the pot values from 0-1023 to 0-15, since there were only 15 haikus to cycle through.

Then I added in the lines of code we had reviewed in our labs. There was a lot of stuff going on to enable serial communication and map the correct values but I felt like I understood what each line of code was doing. I used the function print(newData) and was able to see the values change from 0-15 as I twisted the pot!

Except there was one problem: I couldn’t figure out how to write a command that told the computer that the newData values being received from the serial port should be equal to the variable p I mentioned above.

I tried writing a new function getHaiku(newData) that would display each haiku based on the pot number but it didn’t work. I felt like I tried everything, from creating new variables to setting p = newData, but I could not figure out how to get the pot value to control the animation.

Here’s my full code in case anyone has an idea of what I’m doing wrong:

 

Water elevation in Lake Powell.

85521.ngsversion.1422286517028.adapt.676.1Photograph by Michael Melford, National Geographic Creative

I’ve been living in Utah for the last six years, give or take, and my friends and I have spent a lot of time exploring southern Utah national and state parks.

One of the most iconic bodies of water in the region is Lake Powell, a reservoir on the Colorado River that straddles both Utah and Arizona. Lake Powell is best known for its orange-red Navajo Sandstone canyons, clear streams, diverse wildlife, arches, natural bridges, and dozens of Native American archeological sites.

Since its 1963 creation, Lake Powell has become a major destination for the two million visitors it attracts annually. You can see why we love spending time there:

10590510_10154540007750624_787829644479667926_nPhotograph by my friend Kelsie Moore.

10532870_10154540023060624_3323817325450871258_nPhotograph by my friend Kelsie Moore.

Lake Powell is the second-largest man-made reservoir in the U.S., storing 24,322,000 acre feet of water when completely full. The lake acts as a water storage facility for the Upper Basin States (Colorado, Utah, Wyoming, and New Mexico) but it must also provide a specified annual flow to the Lower Basin States (Arizona, Nevada, and California).

Recent drought has caused the lake to shrink so much, however, that what once was the end of the San Juan River has become a ten-foot waterfall, according to National Geographic. As of 2014, reservoir capacities in Lake Powell were at 51% and the nearby Lake Mead was at 39%.

Drought has really reshaped the Colorado River region. According to the U.S. Drought Monitor, 11 of the past 14 years have been drought years in the southwest region, ranging from “severe” to “extreme” to “exceptional” depending on the year. You can see how drastically the landscape has changed over the past decade by taking a look at this series of natural-color images taken by a the Landsat series of satellites.

This week in ICM, we’re learning how to use objects and arrays in javascript. I wanted to produce a simple data visualization that displayed historical data about the water elevation in Lake Powell since its creation in the 1960s. I also knew that I wanted to use some kind of organic sound in the visualization, exploring p5.sound library.

See the final visualization here.

Screen Shot 2015-10-08 at 11.59.52 AM

I found a database online that contained the information I needed and I created a CVC file that detailed the year in one column and the elevation values in another column.

At first, I envisioned an animated visualization that snaked across the screen and split into fractals as you cycled through each year in the database. I liked the idea of having the design mimic the structure of the Colorado River. Here was my initial sketch:

FullSizeRender

I started playing around with the code and was able to produce an array of values from the CVC file. For instance, I created an array “elevation[]” that pulled the water elevation value for a given year. I wrote some code that allowed me to cycle through the years:

Screen Shot 2015-10-08 at 12.01.54 PMScreen Shot 2015-10-08 at 12.02.06 PMScreen Shot 2015-10-08 at 12.02.23 PM

After getting the years to cycle chronologically, I made an animation of a white line moving across the screen. For each new year, I wanted to draw a bar extending from the white line that helped visualize how the water levels were changing from year to year.

I created a function Bar () and gave it some parameters for drawing each of the bars.

Screen Shot 2015-10-08 at 12.06.52 PMScreen Shot 2015-10-08 at 12.06.59 PMScreen Shot 2015-10-08 at 12.07.31 PM

After defining the function, I started the animation by typing bar.display () with the specified parameters under function draw (). The bars were now a new object.

Next, I wanted to add sound to the visualization. I thought about a few different organic sounds: rainfall, rivers flowing, thunder, etc. In the end, I found a field recording of a thunderstorm in southern Utah and I immediately fell in love with the sound.

Every time a new year started, I introduced a 20-second clip of the sound so that over time you can hear the rolling thunder. I added some brown noise to sit underneath the sound file and some oscillation effects.

Screen Shot 2015-10-08 at 12.16.06 PM

When a new year starts, a new sound file plays, layering over the last sound. When the visualization finishes, the sound disconnects.

Screen Shot 2015-10-08 at 12.16.14 PMScreen Shot 2015-10-08 at 12.16.21 PMHere’s a video of the visualization:

Overall, I liked how this sketch turned out, but I had some major problems with this visualization.

First off, I think that the data I obtained (water elevation values by year) told a much less dramatic story than I had expected. I realized as I was doing research for this blog post that during droughts, it’s not the reservoir levels that experience the most dramatic decline, but the outflux of water is reduced significantly. I think that if I were to do this project again, I would have spent more time researching the data set I wanted to use.

Second, I really didn’t love the simple animated graph I produced. Yes, it told the story in a straightforward way, but I really wanted to produce a fractal/river shape that was more visually compelling than just straight lines. I couldn’t figure out how to do it in time so I might try doing it for a future project.

I think that adding the sounds made this visualization much more interesting and I want to keep exploring the p5.soundLibrary for future sketches.

 

 

Junk food game redux.

Screen Shot 2015-10-01 at 1.34.33 PM

This week’s assignment was to improve upon a previous project by cleaning up and rewriting some of the code for the project. I was excited to spend more time working on my junk food game I built with Jamie last week because there were still a number of things I thought I could improve.

Here’s the updated version of our game.

Before I’d done too much work on revising the code, Jamie told me she’d done some work and had (amazingly!) moved some things around and cleaned up a lot of the code we’d written last week. I added some cosmetic changes to the game, including a change in the animation that occurs when the mouth touches food.

Overall, here are some of the major things that were fixed in the new code:

 

  • One of the reasons the code from the original game was so long was because for each piece of food, we’d had to write code with directions. In the updated version, Jamie defined a new function so that each piece of food had a set of consistent parameters. The new function looked like this:

function Food(f, s, p, ip) {

For each piece of food, f = load the image, s = speed of fall, p = points if it hits the mouth (either +10 or -10) and ip = if you hit the poop. The function was defined at the end of the code after function setup() and function draw():

Screen Shot 2015-10-01 at 1.10.24 PM

In function setup() we’d already established what each of these parameters would be for each piece of food:

Screen Shot 2015-10-01 at 1.10.06 PM

That helped simplify the code considerably, since we could just write a general formula for the entire game rather than for each piece of food individually.

  • In the older version of the game, there was a slight jitter when you clicked the initial button. The jitter was fixed by changing the function from ‘mouseIsPressed’ to ‘mousePressed.’ This seemed to fix the problem altogether.
  • Last week, we struggled to change the final “end” screens so that the screen cleared once the game finished. When you win or when you eat the poop, there are two different screens that are displayed. Here’s what the screen looked like last week:

Screen Shot 2015-10-01 at 12.59.41 PM

We updated the code to fix the error (with some added vulgarity because this is just for fun anyway):

Screen Shot 2015-10-01 at 1.06.50 PM

I thought overall that we made some good improvements to the code without changing the structure of the game overall. I’ve got to give Jamie a lot of the credit for simplifying the code and giving a much stronger architecture to the game we had built. I really want to keep improving the game!

Raw footage from our film project.

For our video assignment, our group decided that the piece we had initially decided to film might prove too difficult to complete in the timeframe we had. We chose to scrap the idea altogether and start again from the beginning.

Katie suggested that instead of doing a documentary-style piece we should focus on a simple narrative piece inspired by the Nico song “These Days.” It’s a melancholy song about a woman whose lover has left her and the ways in which that loss colors her daily life as she walks around the city. We were interested in exploring the ways in which the environment in which we live comes to mirror our internal emotions. I’ve discussed with friends the ways in which New York City has this remarkable ability to reflect your feelings back at you so that you are constantly being confronted by yourself.

With that in mind, we mapped out our storyboard and shot list. We knew that there were some locations we were set on filming: Coney Island, a coffee shop, a corner grocery store, etc. We also knew that new ideas would reveal themselves to us along the way and so we started the shoot with an open mind.

I should add that Naoki and Katie decided I should play the part of the young woman in the film (gahhhh). Here are some highlights from the footage:

Screen Shot 2015-09-29 at 8.53.33 PM

lol_im_a_hod_dog_catch_up_me

cafe

woder_wheel

Overall, I think that Naoki, Katie, and I got fairly adept at adjusting the settings on the camera. We were able to adjust the aperture and ISO in a hurry by the end of the day. I felt so much more confident by the end that I could use the camera well and that we had gotten some really good shots, especially at Coney Island.

It was difficult to capture all the scenes we wanted in one day so we are planning to record some additional footage this week. We have a good sense of the narrative structure, but one goal for this week is to nail down, shot by shot, what the film will look like.

Interactive tech in the NY subway system.

10

Last week I was having a conversation with a classmate about how it seems like New York City has failed to integrate technology into its transportation system compared to other cities. I spent a few weeks visiting my parents in Tokyo this past summer and became very well acquainted with Tokyo’s metro system and the ways in which technology aids travelers. For instance, all passengers use a universal “Passmo” card that scans instantly at kiosks and allows for instantaneous transactions. The Tokyo metro stations all have detailed instructions about how to navigate the city on the metro. There’s also free WiFi everywhere.

For this reason, I was thrilled to see the installation of new “on the go interactive wayfinding kiosks” in many of the New York subway platforms. The kiosks were designed by Control Group, which partnered with the MTA to improve the city’s transportation information.

I decided to observe passengers interacting with the kiosks at the West 4th subway stop in the East Village. In terms of interface, the kiosks have a giant touch screen that displays neighborhood maps, visual directions based on real-time train information, and countdowns to train arrivals.

Before observing the kiosks, I assumed that most passengers would use the kiosks to see when the next train would arrive. From my observations, however, I saw that most passengers didn’t typically use the screens. Those who did appeared to be tourists or visiting from other countries. I noticed that the kiosks were used most often when trains were running late or running on an alternate route and passengers were seeking additional information.

The interface at the kiosks is extremely visual rather than text-heavy, which is conducive for people who might not be familiar with English. I didn’t notice people experience difficulties using the kiosks, although the information about train arrival times were not always accurate. People sometimes appeared frustrated by the lack of accurate information.

On average, most people used the kiosks for about 10 seconds to check the status of a particular train. Here’s a video the MTA made to promote the new installations:

Overall, I thought the kiosks were a brilliant move on the part of the MTA. It’s clear that the New York subway system needed more transparency in terms of information availability and ease of use. According to Chris Crawford, interaction is a cyclic process that requires two actors, human or otherwise, who alternately listen, think, and speak. In this way, physical interaction can be considered a kind of two-way conversation. In this regard, the kiosks are truly interactive.

 

 

Measuring cumulative rainfall with a rain gauge.

IMG_2249

I’ve spent about six years living in Utah, where the climate is arid and drought is a constant concern. According to the U.S. Drought Monitor, about a quarter of the state continues to experience severe (D2) level drought. The region often doesn’t receive the rainfall it needs to keep its reservoirs at capacity.

In keeping with my interest in humans’ relationship with their physical environment and ecological processes, I decided that I wanted my next project to collect information about precipitation.

Using a rain gauge that my friend Joao Costa had lent me, I was able to measure the accumulation of rainfall over time.
IMG_2231

A rain gauge is a self-emptying tipping bucket that collects and dispenses water. It allows you to display daily and accumulated rainfall as well as the rate of rainfall. The gauge essentially acts as a switch, making contact when a specified amount of water enters the bucket.

Rain collects at the top of the bucket, where a funnel collects and channels the precipitation into a small seesaw-like container. After 0.011 inches (0.2794 mm) of rain has fallen, the lever tips and dumps the collected water. An electrical signal is sent back to the Arduino where the digital counter can record what’s called the “interrupt input.”

IMG_2233

Once I decided that I wanted to measure the precipitation using the rain gauge, I did some research into the particular gauge I was using. According to its datasheet, the rain gauge connected to an adaptor, which then only connected to two center conductors. I hooked up the rain bucket like so:

IMG_2245

The thing that is really powerful about the rain gauge is that it can measure the cumulative rainfall over a period of time. I decided that I wanted to connect the LCD screen to the Arduino as an output in order to display the amount of precipitation.

Connecting the LCD screen was very difficult because of the number of wires that needed to be connected to the screen (eight!). The LCD screen also required that I set up a potentiometer to control the brightness of the screen, so I added that to the breadboard.

IMG_2250

Once the setup was complete, I wrote the Arduino program that would display the precipitation information I wanted. I figured out how to set up the LCD screen to display text on a single line.`

Next, I needed to figure out how to display the actual amount of precipitation that had fallen into the bucket. To do so, I created a variable “rainTipperCounter.” Every time the see-saw in the gauge filled with water and tipped, the count went up by one.

Screen Shot 2015-09-29 at 7.49.16 PM

I knew that each time the count increased, 0.011 inches of rainfall had collected in the rain gauge. I programmed the LCD to display the rainTipperCounter, multiplied by 0.011, so that the actual amount of accumulated precipitation was displayed.

Screen Shot 2015-09-29 at 7.51.36 PM

IMG_2247

And just like that, I’d set up a simple rain gauge that tracked cumulative precipitation. It wasn’t raining outside today so I had to test the switch by pouring water into the gauge. Here’s how the final product turned out:

Measuring cumulative rainfall with a rain gauge from Rebecca Ricks on Vimeo.

Junk food heaven. Or, my first game in p5.

Screen Shot 2015-09-24 at 1.49.33 PM

This week’s assignment for computational media was to build something that uses (1) a button or slider and (2) an algorithm for animation.

I immediately knew that however this project played out, I wanted it to involve both junk food and the Beach Boys.

Play the game here!

I was assigned to work with Jamie Ruddy. Jamie and I decided within the first 30 seconds of our conversation that we wanted to build a simple game using interactive elements and animation. Because both of us were drawn to the playful Japanese emojis that we use when we text, we designed a simple game in which the player moves a tongue that catches various pieces of food: hamburgers, apples, eggplant, etc. The player gains points for eating healthy food and loses points for eating junk food.

Here’s the initial sketch I drew:

IMG_2135

There were a lot of steps to build this game. First, we had to create a welcome screen that changed to the game screen once you clicked a button.

Screen Shot 2015-09-24 at 1.32.29 PM

Then, we had to create an assets library of PNG images to load into the game. Each piece of food would drop into the screen where the x-coordinate was random and the y-coordinate started at 0 and increased at each frame.

Screen Shot 2015-09-24 at 1.57.28 PM

Then came the most difficult step in the entire process. In order to animate the emojis and make them interact with one another, we had to create an object. This was a first for me, so I spent a lot of time reviewing the p5 reference library’s explanation of how to create an object.

Within the object, we created a function Food() that displayed each item of food, made it drop at a different rate than the other food, and then checked to see if it had touched the tongue. It also checked to see if the food had hit the bottom of the screen (y=height); if yes, then start the food again at the top of the screen (y = 0).

From there, we needed to ensure the score was increasing/decreasing based on which object the tongue touched. For instance, the apple would +10 and the hamburger would -10. Get to 50 points and you win.

Screen Shot 2015-09-24 at 1.23.28 PM

And for a bonus surprise: Eat the poop emoji and you just straight up lose the entire game (because, gross).

Screen Shot 2015-09-24 at 1.22.29 PM

To make the game more fun, I added the Beach Boys’ “Kokomo” to the sketch. It was simple to call the mp3 in the library by using the preload() function.

Screen Shot 2015-09-24 at 1.15.51 PM

Success! There’s a lot more I’d like to do with this game. For instance, I’d like there to be a location-based animation whenever the food hits the tongue  (right now the screen flashes white).

Here’s a video of how the game works:

 

 

Homelessness in Greenpoint: Initial storyboard.

The homeless population has always had a persistant presence in New York, but in the past several years homelessness in the city has skyrocketed. According to recent reports, the number of homeless in city shelters has risen from 53,000 to 60,000 during mayor Bill de Blasio’s term.

According to de Blasio, people are increasingly losing their permanent housing due to a perfect storm of plateauing income and a hot real estate market reaching into new neighborhoods, like Bushwick or Greenpoint, where landlords once accepted homeless families and their subsidies. Simply put, many families can’t simply keep up and end up in the shelter system.

With that in mind, our group is interested in examining how the homelessness crisis is playing out in one particular street corner in Greenpoint, Brooklyn: the intersection of Milton and Manhattan.

There are two churches on the street, one of which sponsors a daily food pantry for the neighorhood’s homeless population. Alongside the lines of homeless people, however, are the vestiges of gentrification: designer coffee shops, a CVS, boutiques, and a bank. This is just one of the battlegrounds where New Yorkers are increasingly losing their housing and entering the shelter system.

Here is the initial storyboard for the 3-5 minute film we will make:

 

68B90310-BF72-4855-AC85-30BD19FEE632

E33EB94B-E294-4BB1-B5D7-2ECC025748F6

BE5E117C-8FDB-4931-B0D7-29BB8168A2F2