D, I & E Reflection

A very big thank you to Stephen Barrass. I very much value the intensive delivery mode of the Digital Design technology units where I feel like I can quickly progress by implementing and then expanding upon learning immediately - before it can be forgotten and then require recapping as in other potential delivery modes. As with previous Digital Design units, Design, Interaction & Environment has opened up a whole new world.

I have never before played with electronics or microprocessors, let alone felt audacious enough to attempt an interactive installation. Of course with any newly learned skills I will need to keep practicing, and it can sometimes be difficult to find the initiative to do this - however I believe that the rapid progression through this unit will now make it more likely that I have the confidence to attempt a project of complexity sufficient to be inspiring. In fact, if I have time, I would like to set up an interactive installation for the my end of year architecture graduating exhibition, which tends to be well attended and a bit of a party.

A particularly appreciated learning experience I think was the production of an installation, which in many ways was a 'real' project. Group work can be very stimulating, but can in a University situation be more often than in a workplace challenging because of unclear roles, lack of (or competing) leadership and group members who don't pull their weight. Thankfully this unit, as a whole of class group project led and facilitated by Stephen, proved to be very productive, and group members who had diverse backgrounds were able to contribute in different ways.

The best bit of the realness of this project was that Stephen left in all the messy bits: we were able to develop a design direction collectively, which is not the easiest processes; we had to shop for the components and materials, which were not always available meaning that we had to adapt the design; we had to investigate and learn new fabrication technologies; and ultimately we ran out of time to finish within the intensive class time! All wonderful lessons in production.

So through this unit I learnt some technical skills like constructing simple electronic circuits and drawing circuit diagrams, and working with various potentiometers and a microprocessor. I also gained great confidence in the adaptability of my programming, particularly in learning Arduino code, which is slightly different to Processing, and in figuring out how to make a library when I was stuck trying to implement classes.

More importantly perhaps were considerations of pertinent content, meaningful interaction, and legible interfaces and environmental responses in the design of installations. Although we were only able to include in our project some of these design and theory ideas, and perhaps in a fairly limited way, I found the class discussion on background readings and research for project proposals most engaging and worthy of expansion in future classes.

I am looking forward to continuing to work on our installation over the next couple of months. I hope that it can still be somewhat collectively curatable - that is I think that more than one person will be able to fit on the tiles at the same time and so suggest that a focus should be interaction between responses to multiple footfalls.

The theme of exploring self organisation is also something that I have had an ongoing interest in. I like very much the tactility of seeing and interacting with self organising systems that are rendered outside of the computer screen.

R&Sie(n) 'I've heard about'

R&Sie(n) 'I've heard about'

Taking research at the intersection of self organising systems, architecture and physical computing to a conceptual extreme are R&Sie(n), whose project 'I've heard about' for example envisions an architecture not centrally controlled, self constructed and continually grown and adapted, by landscape secreting 'Viabs' which respond to local conditions including chemical emissions data of it's human inhabitants. This biostructure no longer seems quite so futuristic.

D, I & E Intensive 2 Day 3

We now have the projects substantially finished, but we didnt get to install them today or test how they interact or give them individualised behaviours. We will get together for another workshop in the holidays and do a test install then before the exhibition at the Belconnen Arts Centre. I suppose that is an important lesson in just how long all the little things take to get done, and how often things dont work smoothly as expected and so need resourceful on the spot thinking.

I do feel like I have come a long way in 6 days, and have confidence now to finish my tile myself - and then go on to do any other project on my own. I had never worked with electronics before this.

SOFTWARE

I wrote my first library! What a challenge - after spending the evening before adapting our code to classes as I would set them up in Processing, I spent almost the entire morning figuring out how to make them work in Arduino as a library.

Code with library header and class files in foreground

Essentially Arduino is built on C++, which has similar syntax to Processing which is built on Java - but they are not the same or directly compatible. I will have to read up more about this. The previously linked to tutorial was very helpful and got the basic setup for me.

However after hours of debugging and browsing the Arduino forums I still had one unsolvable problem - I couldn't get the constructor to recognise instances of custom classes. I had made seperate led and light sensor eye classes, and was trying to get the led class to know about their own eye as well as the other 3 eyes in the tile. So I got around this by unifying the classes in a single cell class. Lesson - try to make two classes talk to each other at your own peril!

There are a few other key differences between this and a Processing implementation:

1. the code is written in separate files that are saved in the libraries directory of the Arduino install directory (Arduino/libraries) and edited in a text editor such as notepad, these are:
1. Cell.cpp, the class file where all the code goes
2. Cell.h, a header file which is like a contents
3. keywords.txt, a list of keywords to highlight the code in Arduino
2. there is additional wrapping syntax before the constructor and other functions in the class file (ie Cell : : )
3. there must be a link to the standard Arduino library (ie #include "WProgram.h" ) and any other libraries you are using (if you can get them to interface!)
4. the header file lists the constructor and all the parameters and functions, so is a handy reference to remember the names you can call - public functions and parameters can be called, manipulated, updated etc in the same way as you would with Processing classes - private functions and parameters can only be called from within the class
5. in your Arduino code you need to import the library (ie #include <Cell.h> ) and remember to restart Arduino to get it to recognise a new library

It should be easy to hack the library as the code resembles Processing - just remember to add new parameters and functions to the header file. Perhaps adding a timer to keep track of the time that the light has been off would be useful.

However I have tried to design the library to be flexibile so that for most of the things we might want to achieve with the tiles we shouldnt have to touch it. There are some global variables that can be updated via the cellSettings() function. I have added parameters to control the eye flag threshold for turning on/off the LEDs, change the delay between the eye flags and the LEDs turning on, and to set a minimum time that the LEDs have to be on before they can be turned off. I have also added a boolean switch to turn on/off the fade and a time step to lengthen the time it takes to fade.

There is lots that can be done from outside the library - for example the knob is hooked up to change the flag delay via the cellSettings() function and the buzzer is set up with all of it's parameters (frequency, duration etc) outside the library and is hooked up to make a tone when it recieves a toneSet signal which is sent when a cell turns on the lights in the function doLight(). The pressure sensor controls lights differently, outside of the library ( ie not in doLight() ), by simply setting brightVal to be 255. The LEDs can be set at anytime to be any gradient of brightness by changing brightVal.

Also I gave each of the cells specific names (topLeftCell, bottomRightCell etc) not because it matters which way the tile is oriented, but so that it would be easy to program communication between cells remembering which cell is clockwise or anticlockwise adjacent or diagonally opposite. Internal communication may need to be considered by the whole class because with the current code and physical circuit setup ripples will not propagate beyond the tiles immediately adjacent.

Anyway the main outcome that I was trying to achieve was robust base code that has plenty of room for others to develop some differentiated individual behaviors for their tiles.

HARDWARE

Concept diagram showing neighbour communication paths

Circuit diagram showing inputs on left and outputs on right

With limited materials we put together a couple of working protoypes. Richard Spellman and I worked together. There were a few issues that we met.

The red LEDs we have been using are not very strong so we are back to planning to seperate neighbour communication LEDs from display lighting. However we have almost run out of pins on our Arduino microprocessor - there are still a few digital pins, but these can only be ON or OFF. The breadboard is also very busy and we ran out of wires so have confused colours (eg we put the knob on the output side and used black, blue and white wires for ground). It would be nice to have multiple breadboards, and particularly breadboards with positive and ground rails (this would save a lot of wiring).

The piezo pressure sensor creates it's own current so doesnt need connecting to positve. It needs to be wired accross a resistor to register differences in current and we had to change to a 1 mega ohm resistor so that it would be more sensitive. The piezo only creates a tiny amount of electricity so we need a big resistor to draw out the time it takes for it to flow to ground. The piezo we are using is salvaged from a piezo speaker.

Each light sensor responds markedly differently and so will need individual calibration. Also we need to consider if we want to access the knob from outside of the tile casing.

When we setup the circuit in the casing we will need longer wires for neighbour communication light sensors and LEDs, and perhaps other things too, and not everything will be on a bread board. We will have to be careful fixing components down and will need to remember to case exposed wires so they dont short circuit.

We still need to test the audibility of the buzzer through the acrylic and whether we need some pin holes. We also have to prototype different display LED configurations to test the brightness and how they display through different backing sheets, patterns etc - this is something that we will each be doing now individually.

A working prototype Richard Spellman and I put together

The prototype as it fits in the casing - the pressure sensor will sit in the middle just under the acrylic tile, and LEDs and light sensors will be in each cell 

Some longer wires for the LEDs and light sensors prepared by Richard

Stephen Barrass, Natalia Lopez and Nathan Evans drilling the casing together

Also an over site - full credits for the project! Our lecturer is Stephen Barrass and the class is Vanessa Wang,  Anaer Anaer, Nathan Evans, Subyeal Pasha, Natalia Lopez, Amber Standley, Richard Spellman and myself.

D, I & E Intensive 2 Day 2

10AM

In thinking about the design approach to my tile, I have tried to establish a hierarchy of inputs and outputs to assist clarity.

The primary input I think is the pressure sensor (a footstep) as this is the direct interactive input from the audience, so output responses to this should be dominant over other outputs - and immediate. Probably all responses to inputs should be fairly immediate, else they loose legibility. So therefore responses have to either be short, additive or interruptible - rather than queued.

There are 4 neighbour communication channels (inputs/outputs). These must be treated equally I think so that the tile is robust enough to be placed anywhere. For example if a tile is placed in the bottom right corner where it has no right neighbour then it would be silly to have a response behaviour that can only be turned on with a signal from the right and equally it would be silly to try to pass output signals only to the right. Therefore  I think that all neighbour outputs should always be the same signal and that the relevant neighbour input signal states are limited to: 'I am recieveing a signal from a neighbour', 'I am recieving a signal from more than 1 neighbour', 'I am not recieving a signal from a neighbour' and 'I have not recieved a signal from a neighbour for x period of time'. In a 9 square grid - 3x3 - there is 1 centre square that has 4 neighbours, 4 squares that have 3 neighbours and 4 corner square that have 2 neighbours.

Outputs are a bit of a balancing act - between variety of responses and legibility. I could have simply on/off, which might be a bit boring (?) but very clear. At the other extreme, given that we are using analogue signals, I could have continuous gradients to give numerous variations (0-255), but these would likely be harder to distinguish. I am leaning toward steps - perhaps 3-5 to be determined by protyping (eg 3 steps would be LOW: 0-85, MID: 85-170 and HIGH: 171-255).

The output responses will be various combinations of light and sound, including varied brightness, blinking, delays, duration, and frequency, and may be thought of as calls that can be made once or perhaps repeated in a pattern.

For example:

• Pressure LOW   >   Response 1 (OFF)
• Pressure MID    >   Response 2
• Pressure HIGH   >   Response 3 
• Neighbours ALL LOW   >   Response 4 (OFF)
• Neighbours ALL LOW for x time   >   Response 5
• Neighbours 1 x MID   >   Response 6
• Neighbours 1 x HIGH or 2+ x MID   >   Response 7  
• Neighbours 1 x HIGH & 1+ x MID   >   Response 8
• Neighbours 2+ x HIGH   >   Response 9 

That is already 9 different behaviours - at least 6 calls - perhaps even this is too much for clarity?

12PM

Stephen has suggested that as our casing has four quads that we design four pixels per tile. So visually our grid will be 9 square super cells and 36 square sub cells (6x6). We are now planning on passing out different signals on each side (regardless of whether a neighbour exists on that side) and having each pixel behave individually.

Stephen is keen that we first set up all of the tiles to behave in the same coherent way - that is is you step on a tile all four of it's pixels turn on, and then pixels in neighbouring tiles ripple on (each pixel turns on after a second delay if its neighbouring pixel is on). Each pixel has two sides (potentially two neighbours) and only recieves signals from one neighbour and only passess signals to the other neighbour (because each tile has four light sensors and four pixels) - this means that communication paths will be quite particular through the grid.

Stephen has also suggested that we dont need seperate LEDs for communication with neighbours - that leaked light from our diplay LEDs should be sufficient to register differences with the light sensors. We will prototype this and see. The great benefit of less LEDs is that is simplifies our wiring - and we only have limited breadboard space and particularly limited input/output pins on the Arduino microprocessor. We are also concerned that the little red LEDs that we are prototyping with wont be bright enough to make the display engaging - we will have to wait and see how it all comes together tomorrow. We will be able to make changes before the exhibition in Belconnen.

5PM

An update on fabrication - we collected the materials (and learnt how to tie knots to secure them to the roof of the car) and began getting the ply CNC routed in the workshop. The router doesnt cut all the way through so that pieces can be held together stably. Kudos to Nathan Evens who led documentation and supervision of the routing.

CNC router - showing different profile routs and cuts

The acrylic was already cut into 38cm squares for us by Plastic Creations and the ply came from Mitchell Building Supplies as Bunnings Warehouse doesnt keep 25mm ply in stock. We also picked up some non-slip rubber underlay from Clark Rubber.

While I had been working in the fabrication team Vanessa Wang,  Anaer Anaer, Subyeal Pasha and Natalia Lopez had been leading development of the code.

12AM

This evening I was trying to tidy our code with classes but found that they dont behave the same way as in Processing and in fact need to be set up as libraries. Perhaps I can try to implement this tomorrow.

I was also experimenting with different ways to make lights fade off - for example our first code dimmed the brightness by 1 each loop (255 = on, 0 = off). A loop might take up to 15 milliseconds if there is lots of code and I didnt want to change from integers to longs - so my ideas for lengthening the fade time were to either have a counter each loop and only every nth count dim the brightness by 1 or to have a time step and only dim the brightness by 1 after each time step had passed. I went with the time step because I thought it would be more accurate.

Also I was thinking about allowing internal communication via code between the four pixels in a tile about their state (on, off etc).

D, I & E Intensive 2 Day 1

So begins the second intensive for Design, Interaction & Environment. The first intensive we were learning basic skills for Arduino and making some prototype critters that could sense, respond to and influence their environment. This social colony of things were able to interact with their neighbours and displayed some emergent behavior patterns. In this second intensive we are focusing on production of an installation, first for the foyer of Building 9 at the University of Canberra and then perhaps as part of an exhibition in July at the Belconnen Arts Centre.

We spent the morning discussing ideas for the installation and came to a first concept that everyone could develop an individual critter that received environmental input in different ways (laser beam broken, pressure pad in cushion, strokable grass, noise level, Theremin proximity sensor, helium balloons that pull flex sensors, Xbox Kinect movement sensor etc) but was tied together by producing a single note in a coherent soundscape in similar fashion to the ToneMatrix.

We visited the site and found that it was possible to hide things under the terrain/seat (we discovered spiders and dead mice) or behind the acoustic ceiling tiles, and that if we had things that were not too heavy we could velcro them to the green carpet roof/wall or hang them from the light fittings, but that there were not many other places we could safely attach critters. We discussed linking a visualisation on the tv screens and even broadcasting this or a camera feed voyeuristically to other tv screens around the building, and also determined that it would be possible to transmit data by the building's network and so keep our laptops if needed in the server room.

Ultimately however we decided to flip the first concept and make a unified input, as a tiled floor with pressure sensors and our critters under each tile, and have individual, divergent outputs in the form of light and sound. This will essentially be a tidier version of our social colony of things from the first intensive, with the critters still responding to neighbour states but with a more robust communication channel (better aligned, secured and calibrated LEDs and light sensors). The installation will behave and be structured much like a cellular automaton and will feel something like Dance Dance Revolution or the floor of the night club in Saturday Night Fever. As with the critters in the first intensive the light and sound outputs will probably be more of a cacophony than a symphony, but hopefully there will be coherent emergent patterns perceptible.

In the afternoon we set about designing the standard shared hardware and casings. Every tile will have the same case, the same pressure sensor, and the same neighbour communication channels (LED out, light sensor in) to ensure that they robustly fit together and can be tiled in different positions.

We visited the workshop and learnt about using the new fabrication facilities at the University of Canberra - how to set up drawings, that the CNC router can cut sheets up to 25mm deep and 1200mm wide, that routing paths can be set for inside or outside of shape edges and to any depths, and that the laser cutter can engrave by reducing the beam intensity but that depending on the material this will be to different depths.

Stephen Barrass workshopping the casing design

The casings will be 38cm square routed ply with acrylic tops. The hollowed ply will have spaces for the Arduino microprocessor, a battery (we might later try to hook up a power plug) and multiple bread boards (we want to keep the sensors and neighbour interface separate from the light and sound outputs), while maintaining structural support for the acrylic at the sides and centre.

We can get small piezo pressure sensors cheaply - they are a crystalline structure that produce a small current when pressed. The pressure sensor will be placed at the centre of the tile immediately below the acrylic where it will register even visually imperceptible bowing of the acrylic when stepped on.

We will probably use clear acrylic, meaning that individual design decisions in addition to how many LEDs to have, where to place them and how to program them also will include whether to frost or etch the acrylic or back it with translucent paper.

Tomorrow we will finalise the design of the casing and fabricate it. Hopefully we will have some time to design the content too, because the next day we plan to install!

An installation that is collectively curatable

As a class project for Design, Interaction & Environment we will be building an installation for the foyer of Building 9 at the University of Canberra. The site is already perhaps the most 'designed' space at the University with a recently completed refurbishment including an astroturf seat/terrain installation opposite tv screens broadcasting news channels and many walls covered in larger than life prints of significant historical media moments.

Building 9 - two spaces, building foyer with screens and entrance to theatre beyond

Building 9 - astroturf seat/terrain

Building 9 - automatic sliding doors sense movement

Building 9 - stairs adjacent to foyer

The space presents some obvious opportunities as starting points for a new interactive layer of installation. The space is busy with people both passing through and waiting for class or to meet friends. The screens could perhaps be repurposed, the seats could become points of interaction and even the automatic sliding door already has a movement sensor. News media as an already established theme for the space, suggests rich potential additional content including social media sources such as Twitter.

However what really inspires me are projects that make intangible environmental conditions apparent in a poignant way such that a different engagement with the environment is encouraged - I highlighted this in my initial post for this unit with exemplar projects such as Scott Snibbe's Boundary Functions, Daniel Hirschmann's Tuned Stairs and Usman Haque's Sky Ear.

Equally exciting for me is that these projects can be interacted with by multiple agents. Scott Snibbe's project can only display boundaries when there is more than one person, Daniel Hirschmann's project encourages exhibitionism as people walk down stairs and Usman Haque's project visualises mobile phone calls and text messages from the audience. This idea that an installation can be curated by the audience, giving ownership and understanding of process, is powerful. Yet if the installation can facilitate collective curation, that is interactions with multiple 'authors', and be better for it (say because of shared creativity in response to emergent conditions) then it is all the more engaging.

The final important lesson is to keep it very simple. With all of these projects interaction is intuitive and feedback has distilled clarity and pertinence. Vanessa Wang has also highlighted the importance of this.    

Andre Michelle's ToneMatrix is another good project that demonstrates these principles. This week it has been emailed around the architecture studios and I have seen it cause many hours of procrastination - some students have been so fascinated that they have revisited the project a number of times. It is a web version of the Yamaha Tenori-On a synthesiser that allows you to manipulate simple sinewaves by turning on and off pixels in a matrix.

ToneMatrix, Andre Michelle, 2009

As far as proposals go for our class installation, I am interested in a sound based project that can be collectively curated. I believe that sound can cut through the existing visual clutter of the space. So how to realise an interface for a piano stair or giant physical pixel matrix synthesiser like project?

My initial google investigations found that pressure sensitive pads are expensive but that it may be possible to build our own. Vanessa Wang is proposing to work with pressure sensitive pads too. I think that that Stephen Barrass' ZiZi the Affectionate Couch used conductive thread to to make the ottoman sensitive to touch. (Clarification: Stephen says the static electricity generated by stroking the fabric was passed through the conductive thread to an input reader, essentially making the entire surface ottoman sensitive, but that this was impacted by humidity). Perhaps pressure sensitive cushions for the astroturf seat/terrain are a more achievable scale for input than mats to cover the floor, although it may be possible to make piano stairs alternatively with basic lasers aimed at light sensors such that footfalls break the beam. Subyeal Pasha proposes using lasers in a similar way. Amber Standley suggests the use of a Theremin synthesiser which neatly doesn't require any touch, producing sound in response to proximity.

A pixel matrix interface could be constructed with simple on/off buttons and basic light bulbs. It has to be at scale large enough that it can be interacted with simultaneously by multiple people. The experience would be akin to that of operating a switchboard and perhaps therefore explicitly curatorial.

When someone is not curating the installation, perhaps it could keep playing the previously curated state.

Salt Lake City Switchboard, 1914 (Image: Royce Bair)

Both Amber, with the Theremin, and Natalia Lopez, in her experiments with a magnetoPot, are pursuing a sound of (infinite) gradients. In contrast I now notice I have been thinking about a set of on/off switches. Ultimately I am comfortable with either, being more interested to see if we can establish a simple system that through local interactions can display emergent behaviours. The class already achieved this with our fireflys. Perhaps we should try to expand this and turn the foyer into a swamp inhabited by a diverse community of critters?

Vanessa goes further than the idea of curation to suggest game play as a driver of engagement. There is nothing like firing up those competitive juices for getting people hooked - here is a lovely 1994 Wired article exploring the psychology behind why Tetris is so addictive.

Anaer Anaer proposes an opposite focus, that is on mapping environmental conditions (noise).  If this is part of a larger installation then perhaps the output can be a combination of background condition and interaction, and when there is no interaction revert back to mapping background condition.

Design, Interaction & Environment

This is the first of a series of blogs for the unit Design, Interaction & Environment which can be followed with the tag 8200 (the unit number). This unit is being led by Stephen Barrass who has many exciting projects in this field and a particular focus on data sonification (making sound from data). We will be using Arduino and experimenting with inputs from environmental sensors, human interfaces and other data sources, and outputs that have a physical manifestation in the environment such as light, sound and movement. Ultimately as a class we will make an installation for the foyer of Building 9 at the University of Canberra. It is nice to be reassured by Tom Igoe that there is merit in doing another iteration of a theme already well established.

My initial interests in interactivity and environmental awareness stem from a cursory look at some of the following projects as I have come across them, with little understanding of the systems behind them or in general what physical computing is about.


Automation - are buildings too complex for users?

The first and obvious architectural application of physical computing is automation of building systems in response to environmental stimuli - for example adjusting the position of shading or photo voltaic panels based on sun position, turning on/off lights based on occupancy and levels of daylight, adjusting ventilation (opening/closing windows) and turning on/off mechanical heating and cooling systems based on temperature. These computerised controls are broadly referred to as Building Management Systems (BMS) and are often largely unnoticed by building occupants.

The mantra with sustainable buildings, particularly solar passive houses, is that they need active users - windows, curtains etc must be opened/closed at correct times and if they are not, the building can perform significantly worse than a conventional building! Automated systems are common in new commercial buildings and perhaps for this reason could also be important in houses.

A visually expressive example of an automated system is Jean Nouvel's Arab World Institute brise soleil which is a beautiful array of 240 motorised apertures evoking traditional Islamic screens.

Arab World Institute, Paris, architect Jean Nouvel, 1981-87 - the brise soleil seen from interior

A more contemporary example on a larger scale is LAVA's solar umbrellas for the plaza in their winning Masdar City Center design. The solar umbrellas are modeled on flowers and fold up at night.

Masdar City Center design competition winning entry, LAVA, 2009 - solar umbrellas in public plaza

Making the intangible tangible

The next projects are interactive art installations. I, like Vanessa Wang, keep coming back to Scott Snibbe's Boundary Functions. I first saw this project when I was experimenting with abstracted Voronoi diagrams to arrange program and divide space in a second year architecture studio led by Iain Maxwell.

Boundary Functions, Scott Snibbe, 1998

Boundary Functions uses a Voronoi diagram to project personal space as cells with boundaries halfway between people and their neighbours - that is that all space within in the cell is closer to the owner than their neighbour. The cell size, shape and boundary change dynamically as people move and population densities vary. The system uses an overhead camera and projector to track people and display the diagram. I like this project because it makes an important environmental condition that is usually experienced by feeling tangible in a visual way that allows it to be better understood. Personal space is not often thought about critically and this installation makes apparent the contextual nature of it - that is that personal space is not fixed and can only exist in relationship to your neighbours. Snibbe further comments that as the installation can only exist with more than one person and in a physical space, it is a reversal of the usual lonely self-reflection of virtual reality or frustration of virtual communities.

On questioning personal space, this time in relation to machine (can you be intimate with a machine?), I also very much liked a student project that Bert Bongers showed at the 2007 AASA Conference. It was an installation (creature) that had long stick-like limbs and picking up movement or proximity would lean over to be near you (is it leering at me or is friendly? is it invading my personal space? is it going to touch me?!)

Tuned Stair, Daniel Hirschmann 2006, Fabrica exhibition, Centre Pompidou

Another favourite is Daniel Hirschmann's Tuned Stairs - part of the 2006 Fabrica exhibition at the Centre Pompidou. These piano stairs have pressure sensitive pads (mat switches) installed on stairs that (through an Arduino microprocessor!) each trigger a different note. This is a very simple and beautiful  way to make legible the different cadences of people as they move through the space - it also encourages people to perform. Grand stairs in theatres particularly, where patrons are dressed up, but all stairs in public places to some extent, are already associated with performance or exhibitionism. The piano stairs are a contemporary interpretation of giving tuned resonances to architectural elements - an idea that has been resolved by Carlo Scarpa as musical steps in the Brion Cemetery and by Vijayanagara temple builders as musical columns that the priests 'play' most famously in the Vittala temple.

Musical Stairs at Brion Cemetery, Carlo Scarpa 1970-72

Vittala Temple, Vijayanagara 15C

The other project in this theme that I really like, again for it's simplicity, is Usman Haque's Sky Ear which is a floating sculpture of helium balloons, electromagnetic sensors and mobile phones. The changing colours of the balloon LEDs make apparent the underlying electromagnetic patterns and the impact of calls and messages to the balloon phones.

Sky Ear, Usman Haque 2004, Greenwich

Dan Hill in his narrative The Street as Platform further elaborates an understanding of digital substrates - layers of data that can be collected, made legible and interacted with.


The beginnings of a physical computing syntax

From reading a couple of background papers presented at the Sketching in Hardware 2010 conference a syntax, or taxonomy even, for interaction design begins to emerge.

Carla Diana talked about a natural user interface, exemplified by the the gestures used to control the iPhone's touch screen. Emphasising appropriate gestures, screen readability and ergonomics, Diana discussed the position of metaphor, abstraction, mapping, feedback, delight and personality in interface design.

Ellen Do elaborated on this theme,  calling for a human centric view of designing interactions for example

• Hand: touch, press, pick up, hold, squeeze, gesture 
• Body: sight, sound, sense, sit, stand, walk, sleep, movement pattern
• Environment: light, space, ambiance, navigation, context aware.

Do describes a responsive architecture as an environment that takes an active role, initiated changes as a result and function of complex or simple computations, but also knows me and the context it is in and establishes a framework of considerations for designing A-E-I-O-U:

• Activities: What are they doing?
• Environments: Where is it taking place? (eg location, noise, lighting, all the senses)
• Interactions: With whom are they interfacing? (eg talking, seeing, collaborating)
• Objects: What are they using? (eg chair, projector, keyboard)
• User: Who are they? (eg group, individuals, roles, demographic data)

One final project that I want to mention is the interactive white board style hacks for projectors and flat screens using Wii Remotes to track the position of a pen that has an infrared LED. In 2009 Leo Carson and Christine Murray led a group of other MDD students, with support from Mitchell Whitelaw and Gurdev Singh to successfully implement a version in Processing. They ended up setting Boon Jin Goh's Smoothboard, a program that already has more sophisticated features built in, for design crits in the architecture studios. On screen markups that can be recorded as stills or movies finally brings to digital presentations the immediacy and intuition of traditional iterative feedback processes using butter paper. You can see in the video below Gurdev enjoying marking up my project.

Photomontage: the sublime modern; and the generative city - an architecture of remixed geometry

This is a proposal for an interpretive light projection art installation at the partly demolished Cameron Offices. As I wrote in a previous post, the modern architectural masterpiece has been misunderstood and unappreciated, and this project is an opportunity to encourage a new public engagement with the important building.

The intention is to generate a series of photomontages that emphasise the powerful sublime of modern architecture's allusion to ruin, and explore the structural and compositional geometries expressive of this and the ideological urban ordering of public space, landscape and network connections, that is particularly inherent in the Cameron Offices. This perhaps connects to the zeitgeist interest with city form amid contemporary concern about population growth and climate change.


Theoretical Position

Modern architecture alludes to ruin through its expressed structure and lack of ornamentation. The Cameron Offices, by virtue of having never been connected to its intended broader urban network and then by its now partly demolished nature, is a fragment that requires the imagination of what might have been. This allusion to something greater is how the sublime functions, as Edmund Burke and Immanuel Kant determine, and why ruins are sublime.

Giovanni Piranesi, Il Campo Marzio dell' Antica Roma, Second Frontispiece

Manfredo Tafuri in examining utopias through a discussion of Giovanni Piranesi's Campo Marzio demonstrates that excess can reveal truth. Here the 'struggle between architecture and the city, between the demand for order and the will to formlessness, assumes epic tone' and points to 'no other possibility than that of global, voluntary alienation in collective form... a place of total disorder' (Manfredo Tafuri, Architecture and Utopia: Design and Capitalist Development).

Giovanni Piranesi, Carceri d'Invenzione, Plate VII (The Drawbridge)

With other Piranesi drawings, such as his Carceri d'Invenzione (imaginary prisons) series, ambiguity as well as dramatisation and exaggeration contribute to their theatricality.

Gevork Hartoonian proposes ideas of the tectonic and theatricality developed from Gottfried Semper to centre a critical practice within the contemporary culture of spectacle that foregrounds hollow image-making and commodity fetishism. The tectonic, which distinguishes architecture from building, is the purposeful articulation of the relationship between construction (core form) and dressing (art form), and it is this excess that is theatrical - while an art form or image that has no relation to the core form is simply spectacle. See Hal Foster's essay Image Building for a critique of the image-making tendencies of the contemporary architectural avant garde.

Hartoonian suggests that the idea of montage, or the cut, can be used as a critical tool by architects to repress spectacle, and that through cutting, theatricality is activated by the need to think to approach perception of the whole. Hartoonian's writings on theatricality can be found in Crisis of the Object.

This proposal and discussion also recalls Walter Benjamin's wish images that blur dreams and reality, particularly the past or future with the present. The Cameron Offices presents a wish image in that it is now a partly demolished historical fragment but also in that it is a ruin of unrealised ambition. Despite (or perhaps because of) the failure of the modern project, ideologies that point to unattainable ideals are alluring.


Precedent Work

This proposal is indebted to the inspirational photomontages of Beate Gutschow, particularly her S series which I saw exhibited a couple of years ago at the Museum of Contemporary Photography (MoCP) in Chicago. In the S series, fragments of modern architecture are composited out of context in a landscape of vastness where no totality is perceivable to emphasise the sublime and theatrical.

Beate Gutschow, S#14

Along similar lines to Tafuri's discussion of utopia, Philip Gefter in his New York Times review of Gutschow's work reminds that 'optical precision is not the same thing as reality' and that 'fabrication may be truer than life'.

I intend for my montages to reference Gutschow's S series, however even if I didnt, I expect that working with images of modern architecture and emphasising an allusion to ruin a similar feeling would be unavoidable.

A quick google search throws up a few other artists working with related ideas. Adam Ryder also uses digital compositing techniques to explore his interest in urban infrastructure such as carparks. Gregory Crewdson crafts his images in a different way, setting up scenes much like a stage set.

Adam Ryder, Cityscape Photomontage (1/2)

Gregory Crewdson, Brief Encounter

Of course photomontage is not restricted to use by artists and photographers, architects too have a long history of using the technique, particularly for political messages, and it would appear that it is one of the retro presentation techniques currently most fashionable amongst students.

El Lissitzky, Poster for the 1929 Russian Exhibition in Zurich

El Lissitzky, The Constructor (self portrait)

The famous Russian Constructivist architect El Lissitzky pioneered photomontage for use as Soviet propaganda, while more recently Ashton Raggatt McDougall have designed buildings that are literal montages as provocations - their best example is the National Museum of Australia which subversively samples cultural icons to question Australian history and culture.

Ashton Raggatt McDougall, National Museum of Australia - main entrance (the glazing references the Sydney Opera House and the braille on the facade says 'sorry')

Project Description

Like Gutschow's S series, this project will montage images of modern architecture in landscapes of vastness to emphasise the allusion to ruin. The departure from the precedent work is that this photomontage will be generative, dynamic and interactive. Montages will be computer generated from selected source photographic material and will be integrated or overlayed with generated vector cityscapes that remix geometry.

Source material could be limited to John Andrews architecture, architectures that are Structuralist or Brutalist or could include other Australian or international modern architectures.

Imagery will be black and white for a sense of timelessness, and should be relatively high in contrast to show well when projected. I will experiment to be determined whether buildings that are recognisable or fragments that are unrecognisable will produce better effect, and to what extent it is desirable to populate the landscape. I suspect that in Gutschow's S series minimal population is powerful.

The generated vector cityscapes can be potentially outlines, sections or isometric views of buildings or public space, landscape and network connection plans similar to the Nolli Plan.

I can base the generative montage on previous combinatorial systems that I have worked with in Processing such as Monopolise, Space Men, Albers Square Generator and Basic Poetry Mixer. The generated vector cityscapes can be grown with various branching using a model such as the Eden Growth Model, which I have explored in Processing with the Circle Packing Eden Series - see Exploration A, Exploration B, Exploration C and Exploration D.

If text is included in the montage than writing that is descriptive of the sublime qualities of modern architecture would be suitable, such as Gaston Bachelard's The Poetics of Space, Louis Kahn's Light is the Theme and John Andrews Architecture a Performing Art.

Gutschow says she doesnt start with a preconceived image but rather 'sketches' with the component images building a montage very quickly. This gives me confidence that if I can carefully select and prepare the component images I can use computer generation to 'randomly' assemble my montages to reasonable effect. Yet I am acutely aware that when sketching Gutschow employs highly refined intuition based on her theoretical understanding of composition in painting. Computer generated montages could not approach the sophistication of Gutschow's. However with the development of a website interface it would be possible to facilitate sketching that allows the public to follow their own intuitions.

Even with a relatively limited palate of component images there are numerous permutations possible, for example: 5 landscapes, 5 background buildings, 5 focal buildings and 5 foreground objects such as people, has 625 immediately possible combinations, while if each component image can also be flipped/mirrored on just one axis then there are 10,000 possible combinations.

Interaction beyond a website interface, could be facilitated by sound or movement sensors that influence the generation of the vector cityscapes. The Cameron Offices are on the current (until November) trunk bus route between Belconnen and Civic, adjacent to a major stop, providing an excellent opportunity to engage the public.

The major reservation I have about this proposal is that photomontage lends itself to using the building as a giant billboard, as has the previous work of BEAM, which, perversely after what I have said above, means that I will fall to the trap of spectacle, where the art form is disassociated from the core form. The alternate position and exciting opportunity with the Cameron Offices is to use light projection as dressing to enhance the reading and appreciation of the tectonic - to integrate tightly with the building in such a way (involving projection mapping) feels more natural to my architectural instincts. An interesting light projection that takes this alternate position is ToDo's Artificial Dummies which are a flock that avoids and so highlights fenestrations in the wall. Another interesting project is Daniel Rossa's 555 Kubik for the Hamburg Kunsthalle, which reconfigures the buildings geometry.

The development of this project for the Cameron Offices can be followed with the label 8203 (the UC Master of Digital Design unit number).