regulated.lines

If you don’t know Processing, you should.

In Courtney’s Arch448E course last spring (at Maryland) I developed a number or Processing Sketches which made use of a live video feed as a generative device for on screen graphics. These made use of a couple techniques. One technique was sequential frame differencing where each frame (or series of frames) is evaluated against previous frames to determine where pixels have changed. This technique is especially good for situations where light and/or background conditions will change. This was used in my “Studioscapes” sketch, where the live feedback is used to display what is essentially a three dimensional graph of the motion. This became interesting when students began to understand that their motion produced the onscreen effects, which created what I thought was an interesting feedback loop, and often led to complex human patterns of movement. As you can see from the video compilation, the graphics respond not only to the current frame, but decay (logarithmically) over time as a short history of movement.

Studioscape from Benjamin G Callam on Vimeo. 

An alternative technique of static frame differencing worked essentially like a green screen. The program takes an image of the static scene (the background) then checks every subsequent frame for pixel variations from the original. This approach proved difficult primarily because of technical difficulties, but otherwise should be a very easy way to detect onscreen presence (in this case, people). The difficulties arose because the cameras I had at my disposal would automatically adjust their sensitivity as light conditions changed. So for example if someone walked in front of the camera with a white shirt, the camera would adjust and suddenly every pixel has changed (it also didn’t help that I was exhibiting this at dusk)…but at any rate it was frustrating. I was trying this with the built-in iSight in my Macbook Pro, and was not able to find a way to “turn off” the brightness adjustment. Anyhow, for the Particle People project I adapted the Traer particle simulation class to attract to changes found by the frame differencing. This also encouraged strange behavior by the participants at the exhibition (I unfortunately don’t have any footage of) which was heavily influenced by the particle system. 

The feedback loop (in both projects) caused the participants to reevaluate (consciously or not) their local spatial environment with regard to both “real” space and the implied space of the digital environment. In both cases, I was quite surprised by the interest people took in this hybrid environment particularly how common social interactions were transformed by the hybrid environment. Taken as a pair (again unfortunately no footage for the second) this was quite evident.

Virtual Trajectories:

This project explored the virtual proximity of image on the internet. The sketch in this case would visit a news site’s homepage, then based on the available links, randomly select one as the next node to visit. With each visit to a node, the largest image on the page was downloaded and compiled into a super graphic which was the constantly evolving image displayed to the viewer. The juxtaposition of virtually adjacent images (meaning within just a click or two of each other) display were often surprising (if not jarring) and was intended to encourage the viewer to question ideas of image proximity in our contemporary environment. News sites were chosen because their content is constantly changing based on current events (or you might assume), so as the sketch traversed the site, shifts could be detected in the value a particular image might have for that particular news media based on prevalence of image. This sketch was particular challenging for me to write due to my lack of experience with regular expressions, and it was not until I had completed the sketch that I became aware of a preexisting Processing library specifically for link and image traversal. I have not posted the source here yet because I intend to revisit this work and fine-tune the algorithm based on these libraries.

All images compiled from CNN.com

Screen Capture 1

Screen Capture 2

Screen Capture 3

Roto Raster:

A (very) simple script that takes an image and translates it into rectangles (but could easily be anything) based on brightness and outputs the translation as a pdf or DXF. It was created for use in a project I am working on with 3FoldStudio. It’s based on Explode by Daniel Shiffman (check his new book), and was created for easy translation to vector files for a laser cutter,CNC, etc. The same could be done in a number of programs, but hopefully this one will eventually grow an interface to allow for easy web-use. This one could be easily modified to use color, different shapes, etc.

Image before

Image after vectorization

This entry was posted Wednesday, October 29th, 2008 at 6:41 pm and is filed under Architecture, Fabrication, Processing.org. You can leave a response, or trackback from your own site.