D-MARC

This is maybe as close as we will get to a completed version of AR Gilt City. Short of taking the tube to Bank and experiencing it yourself firsthand, this documentation is the best we can offer.

I shot these tests against the night sky.

The new development here is that the player is offered two choices -  to either 'Give Money' or 'Zap' the character. Above charity was shown and the image is restored to full colour. 

Here the 'Zap' option was taken and Bucketman is turned into a fireball, which fades within seconds leaving a vacant space in the night sky.

Here a benevolent player restores the performer...

...here Beggar is incinerated whilst Naked and Bull Fighter are rejuvenated. We are able to statistically see how players have responded to the various characters.

Matthew and I had thought of using smoke instead of the explosion, but it did not function as well.  I do, however, like this rather surreal composition.  The circle is a symbol that Layar uses to mark a position whilst the image loads. 

-- update 2011-06-14; matthew --

An overview of how Layar works is provided in the diagram below.  The phone uses internal GPS and compass readings to work out which direction it's facing in.  In terms of talking to the outside world, the phone first contacts Layar (1) asking for information on the chosen layer, and in response it receives something like a business card with author details and where to go for more information.  The phone then tells a second server where it is (2), and in return is supplied with all of the hotspot locations, graphics etc.

Layar normally uses what is called a RESTful approach.  The server doesn't normally need to save anything - the phone sends all of the information needed on every request.  The server doesn't even care if it is receiving 10 requests from different people, or 10 from the same person.

For our project we wanted each person to have their own collection of points, and so the server has to save information.  The basic process is shown in the diagram below.  When a request is made, if the user has made a request before then their details are loaded from the server; if not then a record is created for them by copying the initial set of points.

By Mark Durden

Most current photographers merely reflect the 'objective' misery of the human condition-- a wretchedness which has apparently become a positive object of perverse desire for so many aesthetes and image racketeers. Just as there is no primitive tribe which does not have its anthropologist, soon there won't be a homeless person sleeping curled up amid his filth who doesn't find a photographer leaping out of the urban jungle to capture on film the eternal sleep of the pauper. 
Jean Baudrillard

Jean Baudrillard's damning attack on the documentary photographer is perhaps a useful way to begin to think about the implications of John Goto's series, Gilt City: full figure 'portraits' of outsider urban types-- street entertainers, homeless people, drug dealers, hawkers. Many of these colourful characters are set against the blank 'grey' corporate architecture of late capitalism, the generic faceless façades of the banking worlds of the over developed West; façades which, in a comic version of modernity, sometimes reflect the rushing masses of suited business types.

What distances Goto from documentary's tradition of picturing misery and wretchedness is his recourse to a staged tableau photography and the use of digital compositing. Friends and associates of the artist-- including critics, curators, students, neighbours-- are dressed and posed by the artist in a studio setting. The portrait is then digitally mapped onto particular spaces in the City of London's financial district. As a result there is neither filth nor degradation in his pictures. Indeed, Gilt City might be seen to continue the ethical disavowal of documentary photography which characterises the work of Jeff Wall. Only Wall's pictures are bereft of the parody and playfulness of Goto's tableaux. And while Wall will meticulously mime a documentary look, Goto never disguises his pictures' artifice. Instead, artifice and theatre is flagged up in Gilt City.

Goto uses the codes of documentary to frame and identify his subjects. Playing out permutations of difference and otherness, his photographic theatre, however, deliberately upsets the familiar patterns of viewing such documentary-type subject matter. The pictures are discordant in this respect. For all the immediate signs and signals of alterity, the people who populate Gilt City fail to cohere as distant and distinct from its middle class audience. As a result, his pictures resist and oppose the simple liberal humanist reflex of pity. They are also bereft of the affective dimension which tends to be associated with documentary practice: the pathos accrued by the photo as index and that sense of the residue of the emotionally charged interaction-- however respectful, however abusive-- between subject and photographer.

Bacchant 2002/3

Details in Goto's staged and knowing tableaux function as signs whose connotations very often flout and undermine the codes and conventions, the rhetoric, ordinarily associated with documentary. There is an attention to signifying details: to do with the style of clothing, incongruously often fashionable and trendy-- evident through all the designer labels and logos-- and the accompanying objects, which are often just as anomalous, like the can of Beck's in his 'portrait' of the urinating man, for example: Beck's, renowned for its sponsorship of contemporary art, being more readily associated with the YBA crowds than the street drinker. The man also carries a copy of Bill Jordan's book A Theory of Poverty and Social Exclusion, implying an improbable self-reflective and theorised view of his own position. All this suggests a conscious playing with the codes and signs used to judge, identify and 'label' people.

Click here to read more

One of the influences on Gilt City is Marcellus Laroon’s “Cryes of the City of London Drawne after the Life” (1687).  In a extensive series of drawings made from observation, Laroon depicts the street people of London’s Covent Garden as single figures, each hustling their wares. The artist shows various levels of poverty through a meticulous attention to detail.

Jacques Callot’s (1592–1635) engravings of `Grotesque Dwarfs’ provided a model for Crown Derby’s pair of ceramic figures known as The Mansion House Dwarfs.  The figures commemorate a father and son who earned a living by frequenting the area around Mansion House in the City of London, wearing advertisements pinned to their hats.

Here is the apparatus I used to film the AR video.  The iPhone slots into the front, and a small camera is pressed into an aperture at the back. Matthew had suggested a cereal box, but in the event I used a Cheese Twists carton. It could do with refining.

New technologies always seem to stir up social anxieties.  Since I was a child TV has been the subject of endless ‘expert’ speculation about its detrimental effect on the young.  Now that we are loosing interest in the medium, similar discussions are applied to digital technologies.

Here is a picture of our new grand daughter, Summer, at five days old, courtesy of her Mum and Dad…and Skype!

Sometimes you get lucky.  I’d been weighting up whether or not to go to the City on Saturday to check out some recent adjustments Matthew had made to our AR project.  Logic told me to wait until a weekday when the streets would be full of office workers that I could incorporate into the installation shots.  As logic is no friend of art, I went anyway and here is what I found.

A film crew of about twenty, with all the gear, were shooting three actors against the façade of the Royal Exchange.  Looking like living sculptures dressed in classical robes, the actors were perched on boxes covered in white cloth.  The crew also looked the part, playing their allotted roles.

At best the setup might have referred to Godard’s classical statues in Le Mépris, or Baptiste’s serenade of Garance in Les Enfant du Paradis - but I had my doubts!

Living Sculpture from Gilt City 2002/3

Anyway, I’ve been interested in living statues for a while and in fact incorporated the one above in the original version of Gilt City. I even once judged the World Living Statues Award in Arnhem, Netherlands!

Avatars; Living Statues; City Workers; Augmented Figures; you get the picture.  I also shot some video, which I’ll post when edited.

I made this after reading recent posts by Shuka Glotman and Gudrun Bielz. 

Actually, I love having information at my fingertips; I’m addicted to it!  I grew up in an informational desert and have spent much of my adult life trying to fill in the gaps.

Yesterday I was reading Lloyd Bradley’s excellent book on Jamaican music, ‘Bass Culture’. The question I’ve been asking myself for some time is how you get from Mento and the influence of American R&B, to Ska?  The acquisition of Trojan’s box set ‘Jamaican R&B’ went some way to answering this as you can hear that shift to the off beat, but Bradley takes you through it one rare recording after another. From ‘Manny Oh’ through ‘Easy Snappin’’, ‘Boogie in My Bones’ and ‘Pink Lane Shuffle’ to the seminal ‘Oh Carolina’ – and all of it is readily available on YouTube. It made for a great afternoon!

In a Parisian street I was moved by the sight of a beggar bent double by her arthritic condition.   I became suspicious, however, when five minutes later I saw a second beggar improbably displaying the same condition. I started to figure out the rhetoric of the pose. They must be young fit women to hold this position, even with the help of the crutch and whatever supports were hidden beneath their baggy clothing. The headscarves concealed their age, and a crucifix confirmed the piety of the pose. Or was I shamefully wrong?

There are a number of artists using AR to make political and social comment.  This morning John-Mark Ikeda sent me a link to a new piece he has created which also deals with the Banking Crisis.

FREE FALL is a multi-site installation art piece using Augmented Reality. It is currently installed at the bank headquarters of the top ten recipients of TARP funds. I notice that John-Mark is using an iPad to good effect.

Other political AR projects I’ve recently come across include -

 An artwork by Tamiko Thiel commemorating the ‘Carnation Revolution’ which broke out in Largo do Carmo (Carmo Square) in Lisbon, Portugal, on April 25th, 1974.

http://www.youtube.com/watch?v=VxBEr_bq_0k

Mark Skwarek’s ‘Liberty to Libya’ project, showing an augmented reality dove which is presently circling the city of Tripoli, Libya.

http://libertytolibya.wordpress.com/

John Craig Freeman’s (of 4Gentlemen) Tiananmen Square AR project commemorates 'Tank Man' and the erecting of the 'Statue of Democracy' by the Chinese student movement in 1989.

http://fourgentlemen.blogspot.com/2011/01/tiananmen-square-augmented-reality.html

My approach to recording our AR installations has been as a photographer, awaiting a significant coincidence of actual event before the lens, with the augmented onscreen image.

To use video is more of a problem as the Layar App doesn’t have a video capture facility, so it’s a matter of using a second camera to shoot the onscreen events.  Surface reflections can distract from the image.  I offer a solution here using a kind of monorail and dark cloth, like in the good-old-days!

In a previous post I mentioned that there were fundamental challenges that could delay AR's mainstream adoption.  The main challange that has existied since the very beginning is termed 'Registration' - the alignment of the real and the virtual.

Although it isn't limited to computer systems.  Consider Pepper's Ghost, which John introduced in an earlier post (http://www2.derby.ac.uk/dmarc/digital-hybridity/augmented-reality-some-precedents.html).  Neither the on-stage actor nor the ghost can see each other.  They must act to empty space.  Only the audience sees the combined effect.  Here the solution is meticulous rehearsal, and the marking of points on the floor.

In computer-based augmented reality the observer takes the place of the audience, and the computer takes the place of the ghost.  Hidden in the dark, 'under the stage', the computer has two main ways of gaining information about the outside world.

Optical Tracking

The computer examines the video feed and looks for patterns that it recognises.  This is the main system that the Nintendo 3DS uses.  It's cheap and accurate, but has a limited field of view.  The further the object is from the camera, the larger it has to be, and anything occluding the object breaks the tracking.  

The image above shows an early example of optical tracking (Mohring et. al., 2004).  Modern systems can identify images (such as DVD covers), but the processing required is similar 

Sensor Tracking

This has the most variety in techniques, mainly because the best solution has yet to be found.  Smart-phones use GPS positioning to identify their location, and digital compases to identify their orientation.  With this information they move and rotate their internal virtual world to align with the real world around them.  However, GPS doesn't work indoors (or particularly well in heavily built-up areas).  There are highly accurate sensor systems that can be used indoors, but they are expensive and have limited range.

The original optimism for AR has been tempered somewhat, but there are still plenty of opportunities to work within the limits of the technology.

The City of London: augmented reality installation, screenshots from iPhone 

Naked, Bullfighter and Beggar

Security

Tout

Tout, Bullfighter and Naked

Dealer

Bucketman

Security

Worker

Naked

Nancy - I thought this picture might be of interest in relation to your post about Courbet's anachronistic notion of history (strangely both realist and allegorical - like some types of digital photography).  The conservationist here attempts to preserve the fabric of 'The Painter's Studio' from the metronomic pounding of material time. Whenever in Paris I revisit it to check out its bitumen shadows, the canvas seams and its sheer expanse. And so I remind myself that it has another life, beyond that of the unconscious.

Courbet's 'Stone Breakers' is in fact more a specter, incinerated by the allied bombing of Dresden. And yet it lacks the dreamlike sense of history which makes ‘The Painter’s Studio’ so much of our times.

Here is Plan A.  We are prioritising a viewpoint, which is beside Wellington's statue on Mansion House Place. Because of the radial arrangement, overlapping images will not occur.

It will be interesting to experience the power relations implied by this central viewpoint. In theatre design the central seat was reserved for the King; in panopticon prison design the omniscient observer; and in games design the gamer.  Here there will be alternative, multi-faceted and layered renderings offered from other viewing positions. 

If the circumference is too small, then Plan B is to situate the piece around Jubilee Park at Canary Wharf.

In the last installment of the infamous Hype Cycle of Emerging Technologies produced by Gartner, Augmented Reality was nearing the peak of hype, soon to plummet, as it fails to meet people's high expectations - and is still 5-10 years away from mainstream adoption.

Of course it's a source of much debate; and anyone with an interest in technology will disagree with some of the points.  In terms of augmented reality I think that they're both right and wrong - in that the point needs to be split into two.

In 2003 when the University of Sydney produced the ARPhone concept, the only way to create augmented reality on a mobile device was to use a nearby PC to do the complex processing.  1 year later the first version of the ARToolkit allowed AR on a mobile device at 4-5 frames per second.

It took seven more years, but with the arrival of the Nintendo 3DS there is now a consumer electronics device launched with Augmented Reality firmly in mind.  I recently got a chance to try it out and was impressed by the quality.  The consumer gluttony of this coming Christmas will likely see both the release of AR game and masses of 3DS's shipped.

The smartphone market has also seen an explosion of AR apps in recent times.  They certainly have the required hardware, but a limiting factor seems to be a lack of software libraries/platforms.  Layar is helping in that respect, but there's room for more.

However, when Augmented Reality was imagined, it was for things like overlaying medical scans on patients, viewing piping and wiring schematics en-situ in buildings, and showing blue-print layouts in complex manufacturing tasks.  For these applications there are fundamental challenges that remain unsolved, and are, if anything, more than 10 years away.

A March to the Bank by James Gillray, 1787

This image is commonly said to be Gillray’s protest against the considerable nuisance caused to pedestrians by a detachment of guards, known as the Bank Piquet, as it marched daily to the Bank of England.

The Piquet was formed in response to an attack on the Bank by a mob during the Gordon Riots of 1780.  Soldiers’ shot on the rioters and a number were killed outside the Bank. Read in this context, Gillray’s satire can be thought to show the defenders of privilege and wealth, trampling the poor.

The drastic architectural practice of reconstruction within retained facades took off in the City in the 1980s, as a way of getting round preservation orders on buildings (see Nikolaus Pevsner & Simon Bradley (1997) London1: The City of London from The Buildings of England series, published by Penguin Books, London). 

An earlier example of this same approach is provided by Sir Herbert Baker’s reconstruction (1921-39) of John Soane’s architectural masterpiece The Bank of England. The building was gutted in what Pevsner described as "the greatest architectural crime, in the City of London, of the twentieth century". Only parts of Soane’s boundary wall now remain.

We have been thinking about visual effects that preceded Augmented Reality: those in which images appear ‘before your eyes’, superimposed onto the real-time actual world.  Any further suggestions would be appreciated.

Filippo Brunallaschi's Pinhole Perspective Device.

 An exact copy of the Baptistery of San Giovanni was painted onto a panel of burnished silver into which a small viewing hole was cut.  Standing just inside the adjacent Cathedral, looking towards the Baptistery, a mirror was raised replacing the actual building with the reflection of the painted one.  The silver substrata reflected the actual clouds in real time. 

(We are indebted to Paul Greco for suggesting this link to AR).

Phantasmagoria

The phantasmagoric effect was developed in C18 French theatre, and involved images projected by modified magic lanterns onto smoke or semi-transparent screens.  The use of multiple projectors, sometimes mounted on tracks, allowed for dynamic traveling and zooming effects, which when set against real actors and objects caused great astonishment.

Walter Benjamin compared the experience of the modern city and commodity culture to that of phantasmagoria.

Pepper’s Ghost

Using real actors, mirrors and glass sheets, chemist John Henry Pepper successfully adapted Henry Dircks’ phantasmagoria for the London stage in 1862, to produce spectacular ghost effects.

Musion Eyeliner

Musion Eyeliner is a modern variant on Pepper’s Ghost which uses high definition video projection or LED screens to allow moving images to appear within a live stage setting. At the 2006 Grammy Awards, Madonna performed with virtual band Gorillaz using Musion 3D holographic projection mixed with live performance. In the first part of this video, her image is projected, then actual.

Watch video

Mattes

Bob Scifo’s matte for The Seventh Sign (1988)

Torn Curtain glass matte by Albert  Whitlock (see more) which is 90% painted

At its simplest a matte in cinematography is a landscape or architectural interior painted onto a sheet of glass hung before the lens, through which a live action scene is shot.  The combined image has the appearance of a continuous space.  We intend to research further this interesting technique, which surely has connections with Brunelleschi’s pinhole perspective device?

Telescopes

Modern versions of seaside telescopes sometimes incorporate AR technology.  This particular project by ARGUI allows you to simultaneously see the exterior and interior of a building.

View video 

Brown’s final speech as Chancellor of the Exchequer at the annual Mansion House Dinner in June 2007 celebrated ‘ London’s position…not only as the international financial centre of the world but of global pre-eminence’ and congratulated himself for avoiding ‘a regulatory crackdown’ after the financial bust at the beginning of the decade.

A. Callinicos, Bonfire of Illusions (Cambridge 2010), p. 76