A Society of Simulations

An interviewer once asked Pablo Picasso why he paints such strange pictures instead of painting things the way they are. Picasso asks the man what he means. The man then takes out a photograph from his wallet and says, “This is my wife!”. Picasso looks at the photo and then says: “Isn’t she rather short and flat?”.

Introduction

This essay aims to increase our understanding of simulations and their impact on our notion of reality. Following on some observations regarding the dominant role of visual representations in our culture, I will argue that we are now living in a society, in which simulations are often more influential, satisfying and meaningful than the things they are presumed to represent.

Media technologies play a fundamental role in our cycle of meaning construction. This is not necessarily a bad thing, nor is it entirely new. Yet, it has consequences for our concepts of virtual and real, which are less complementary, than they are usually understood to be.

Before you read on, a personal anecdote from my youth: when I was a child, I thought the people I saw on TV were really living inside the television. I wondered where they went when the TV was turned off and I also remember worrying it would hurt the TV, when I switched it off.

Obviously, I am a grown man now and I’ve long learned that the television is just a technological device, created to project distant images into the living room of the viewers and that those flickering people weren’t actually living inside the cathode ray tube.

Now I return to my argument. Over the last century or so, the technological reproduction of images has grown explosively. Each of us is confronted with more images every day than a person living in the Middle Ages would have seen in their whole lifetime. If you open a 100-year-old newspaper you will be amazed by the volume of text and the absence of pictures.

How different things are today: the moment you are born, covered in womb fluid, not yet dressed or showered, your parents are already there with the digital camera, ready to take your picture. And of course the pictures are instantly uploaded to the family website, where the whole world can watch and compare them with the medical ultrasound photographs already shared before you were born.

Visual power

We see more images every day than a person living in the Middle Ages in a whole lifetime.

Images occupy an increasingly important place in our communication and transmission of information. More and more often, it is an image that is the deciding factor in important questions. Provocative logos, styles and icons are supposed to make us think we are connected to each other, or different from each other.

Every schoolchild nowadays has to decide whether he or she is a skater, a jock, a preppie, or whatever. Going to school naked is not an option. But no matter which T-shirt you decide to wear, they are inescapably a social communication medium. Your T-shirt will be read as a statement, which your classmates will use to stereotype you.

I remember the strange feeling of recognition I had when I was in Paris for the first time and saw the Eiffel Tower. There it was, for real! I felt as if I was meeting a long-lost cousin. Of course, you take a snapshot to show you’ve been there: ‘Me and the Eiffel Tower’. Thousands of people take this same picture every year. Every architect dreams of designing such an icon. Today, exceptional architecture often wins prizes before the building is finished; their iconic quality is already recognized on the basis of computer models. [1]

Picture this!

Does anyone still remember the days when a computer was a complex machine that could only be operated by a highly trained expert using obscure commands? Only when the graphical user interface (GUI) was introduced did computers become everyday appliances; suddenly anyone could use them.

Today, all over the world, people from various cultures use the same icons, folders, buttons and trash cans. The GUI’s success is owed less to the cute pictures than to the metaphor that makes the machine so accessible: the computer desktop as a version of the familiar, old-fashioned kind.

This brings us to an important difference between pictures and pictures – it is indeed awkward that we use the same word for two different things. On the one hand, there are pictures we see with our eyes. On the other, there are mental pictures we have in our heads – pictures as in “I’m trying to picture it.”

Increasingly, we are coming to realize that ‘thinking’ is fundamentally connected to sensory experience. In Metaphors We Live By, Lakoff and Johnson (1980) argue that human thought works in a fundamentally metaphorical way. Metaphors allow us to use physical and social experiences to understand countless other subjects. The world we live in has become so complex; we continuously search for mental imagery to help us help us understand things.

Thus politicians speak in clear sound bites. Athletic shoe companies do not sell shoes, they sell image. Thoracic surgeons wander around in patients’ lungs like rangers walking through the forest, courtesy of head-mounted virtual-reality displays.

You would expect that this surfeit of images would drown us. It is now difficult to deny that a certain visual inflation is present, and yet our unslakeable hunger for more persists. We humans, after all, are extremely visually oriented animals. From cave paintings to computers, the visual image has helped the human race to describe, classify, order, analyze and grow our understanding of the world around us (Bright, 2000).

Perhaps the most extraordinary thing about our visual culture (Mirzoeff, 1999) is not the number of pictures being produced but our deeply rooted need to visualize everything that could possibly be significant. Modern life amid visual media compels everyone and everything to strive for visibility (Winkel, 2006). The more visible something is, the more real it is, the more genuine (Oosterling, 2003). Without images, there seems to be no reality.

Virtual for real 

When considering simulations, one almost immediately thinks of videogames. Nowadays, the game industry has grown bigger than the film industry and its visual language has become so accepted that it is almost beyond fictional. Virtual computer worlds are becoming increasingly ‘real’ and blended with our physical world.

In some online roleplaying games, aspiring participants have to write an application letter in order to be accepted to a certain group or tribe. We still have to get used to the fact that you can earn an income with gaming nowadays (Heeks, 2008), but how normal is it anyway, that at the bakery round the corner, you can trade a piece of paper – called money – for a bread? [2]

Most people would denounce spending too much time in virtual worlds, but which world should be called virtual then? Simply defining the virtual as opposite to physical is perhaps too simple. The word ‘virtual’ has different meanings that are often entangled and used without further consideration. Sometimes we use the word virtual to mean ‘almost real,’ while at other times we mean ‘imaginary’. This disparity in meaning is almost never justified: fantasy and second rank realities are intertwined.

It would be naïve to think simulations are limited to video games, professional industrial or military applications. In a sense, all reality is virtual; it is constructed through our cognition and sensory organs. Reality is not so much ‘out there’, rather it is what we pragmatically consider to be ‘out there’. Our brain is able to subtly construct ‘reality’ by combining and comparing sensory perceptions with what we expect and already know (Dennett, 1991; Gregory, 1998; Hoffman, 1998; IJsselsteijn, 2002).

Do we still have genuine experiences at all, or are we living in a society of simulations?

Even the ancient Greeks talked about the phenomenon of simulation. In the Allegory of the Cave, Plato describes human beings as being chained in a cave and watching shadows on the wall, without realizing that they are ‘only’ representations of what goes on behind them – outside of the scope of their sensory perception. In Plato’s teaching, an object such as a chair, is just a shadow of the idea Chair. The physically experienced chair we sit on is thus always a copy, a simulation, of the idea Chair and always one step away from reality.

Today, the walls of Plato’s cave are so full of projectors, disco balls, plasma screens and halogen spotlights that we do not even see the shadows on the wall anymore. Fakeness has long been associated with inferiority – fake Rolexes that break in two weeks, plastic Christmas trees, silicone breast implants, imitation caviar –, but as the presence of media production evolves, the fake seems to gain a certain authenticity.

Modern thinkers agree that because of the impasto of simulations in our society, we can no longer recognize reality. In The Society of the Spectacle, Guy Debord (1967) explains how everything we once experienced directly has been replaced in our contemporary world by representations. Another Frenchman, Jean Baudrillard (1981), argues that we live in a world in which simulations and imitations of reality have become more real than reality itself. He calls this condition ‘hyperreality’: the authentic fake.

In summer we ski indoors; in winter we spray snow on the slopes. Plastic surgeons sculpt flesh to match retouched photographs in glossy magazines. People drink sports drinks with non-existent flavors like “wild ice zest berry”. We wage war on video screens. Birds mimic mobile–phone ring tones [3]. At times, it seems the surrealists were telling the truth after all.

And though you certainly cannot believe everything you see, at the same time, images still count as the ultimate evidence. Did we really land on the moon? Are you sure? How did it happen? Or was it perhaps a feat of Hollywood magic? Are we sure there is no Loch Ness Monster? A city girl regularly washes her hair with pine–scented shampoo. Walking in the forest with her father one day, she says, “Daddy, the woods smell of shampoo.” Do we still have genuine experiences at all, or are we living in a society of simulations?

Media schemas 

A hundred years ago, when the Lumière brothers showed their film ‘L’arrivée d’un train’ (1895), people ran out of the cinema when they saw the oncoming train. Well, of course – if you see a train heading towards you, you get out of the way. Today, we have adapted our media schemas. We remain seated, because we know that the medium of cinema can have this effect.

Our brains actually have only limited capabilities for understanding media.

Media schemas [4] are defined as the knowledge we possess about what media are capable of and what we should expect from them in terms of their depictions: representations, translations, distortions, etc (IJsselsteijn, 2002; Mensvoort & Duyvenbode, 2001; Nevejan, 2007). This knowledge enables us to react to media in a controlled way (“Don’t be scared, it’s only a movie.”).

A superficial observer might think media schemas are a new thing. This would be incorrect. For centuries, people have been dealing with developments in media. Think of carrying on a telephone conversation, painting with perspective, or composing a letter with the aid of writing technology – yes, even the idea that you can set down the spoken word in handwriting was new once.

Let’s face it. Our brains actually have only limited capabilities for understanding media. When our brain reached its current state of evolutionary development in Africa some 200,000 years ago (Hedges, 2000; Goodman et al., 1990), what looked like a lion, actually was a lion! And if contemplating the nature of reality at that point would have been a priority, one would have made for an easy lion’s snack (IJsselsteijn, 2002).

Although we do seem to have gained some media awareness over the years, some part of this original impulse – in spite of all our knowledge – still reacts automatically and unconsciously to phenomena, as we perceive them. When we see the image of an oncoming train, we physically still are inclined to run away, even though cognitively we know it is not necessary.

Our media schemas are thus not innate but culturally determined. Every time technology comes out with something new, we are temporarily flummoxed, but we carry on pretty well. We are used to a world of family photographs, television and telephone calls.

Imagine if we were to put someone from the Middle Ages into a contemporary shopping street. He would have a tough job refreshing his media schemas. But to us it is normal, and a lucky thing, too. It would be inconvenient indeed if with every phone call you thought, “How strange – I’m talking to someone who’s actually far away.”

We are generally only conscious of our media schemas at the moment when they prove inadequate and we must refresh them, as those people in the 19th century had to do when they saw the Lumière brothers’ filmed train coming at them.

Media sphere

I once took part in an experiment in which I was placed in an entirely green room for one hour. In the beginning everything seemed very green, but after some time the walls became grey. The green was not informative any more and I automatically adjusted.

Something similar seems to be going on with our media. Like the fish, who do not know they are wet; we are living in a technologically mediated space. We have adjusted ourselves, for the better because we know we will not be leaving this room any time soon.

Today, media production has expanded by such leaps and bounds that images and simulations are often more influential, satisfying and meaningful than the things they simulate. We consume illusions. Images have become part of the cycle in which meanings are determined. They have bearing on our economy, our judgments and our identities. In other words: we are living the simulation.

A disturbing thought, or old news? In contrast to Plato, his pupil Aristotle believed imitation was a natural part of life. Reality reaches us through imitation (Aristotle calls it mimesis): this is how we come to know the world. Plants and animals too, use disguises and misleading appearances to improve their chances of survival (think of the walking stick, an insect that looks like a twig).

Now then, the girl that says that “the woods smell of shampoo”, should we consider this a shame and claim that this young child has been spoiled by media? Or is this child merely fine-tuning herself with the environment she grows up in? In the past, the woods used to smell of woods. But how interesting was that anyway?

Our interfaced world-view

Four centuries ago, when Galileo Galilei became the first human being in history to aim a telescope at the night sky, a world opened up to him. The moon turned out not to be a smooth, yellowish sphere but covered with craters and mountains. Nor was the sun perfect: it bore dark spots. Venus appeared in phases. Jupiter was accompanied by four moons. Saturn had a ring. And the Milky Way proved to be studded with hundreds of thousands of stars. When Galileo asserted, after a series of observations and calculations, that the sun was the center of our solar system, he had a big problem. No one wanted to look through his telescope to see the inevitable.

The concepts of reality and authority are much more closely related to one another then most people realize.

While some dogs have such limited intelligence that they chase their own tails or shadows, we humans like to think we are smarter; we are used to living in a world of complex symbolic languages and abstractions. While a dog remains fooled by his own shadow, a human being performs a reality check. We weigh up the phenomena in our environment against our actions to form a picture of what we call reality. We do this not only individually, but also socially (Searl, 1995).

Admittedly, some realities are still rock solid -– simply try and kick a stone to feel what I mean. However, this is not in conflict with the point I am trying to make, which is that the concepts of reality and authority are much more closely related to one another then most people realize. Like the physical world, which authority is pretty much absolute, media technologies are gradually but certainly attaining a level of authority within in our society that consequently increases their realness.

Today the telescope is a generally accepted means of observing the universe. The earth is no longer flat. We have long left the dark ages of religious dogma and have experienced great scientific breakthroughs, and yet there are still dominant forces shaping our world-view. As we are descending into the depths of our genes, greet webcam-friends across the ocean, send probes to the outskirts of the universe, find our way using car navigation, inspect our house’s roof with Google earth and as it is not unusual for healthy, right-minded people to inform themselves about conditions in the world by spending the evening slouched in front of the television, we come to realize that our world-view is fundamentally being shaped through interfaces.

Surely, the designers of these interfaces have an important responsibility in this regard. As media technologies evolve and are incorporated within our culture, our experience of reality changes along. This process is so profound – and one could argue, successful – it almost goes without notice, that to a large extent, we are living in a virtual world already.

Written by Koert van Mensvoort, published in What you see is what you feel. PhD Thesis, Eindhoven University of Technology.

ISBN: 978-90-386-1672-8 (Download PDF)

Notes

1. Examples of architectural structures that are already famous and celebrated before being build are the Freedom Tower by Liebeskind/Childs in New York and the CCTV building by Rem Koolhaas in Beijing.

2. We usually do not realize that ‘money’ is in many respects a virtual phenomenon: a symbolic representation of value constructed to replace the awkward, imprecise trading of physical goods. Indeed, paying $50 for a pair of sneakers is much easier than trading two chickens or a basket of apples for them. As long as we all believe in it, the monetary system works fine.

3. The Superb Lyrebird living in Southern Australia sings and mimics all the calls of other birds, as well as other sounds he hears in the forest – even cellphone ring-tones, chainsaws and camera shutters – to attract females (Attenborough, 1998).

4. The term media schemas stems from the concept of schemas, which in psychology and cognitive sciences is described as a mental structure that represents some aspect of the world (Piaget, 1997). According to schema theory, all human beings possess categorical rules or scripts that they use to interpret the world. New information is processed according to how it fits into these rules. These schemas can be used not only to interpret but also to predict situation occurring in our environment.

References 

– Attenborough, D. (1998). Life of Birds. DVD from the BBC Series, 2 Entertain Video, ASIN: B00004CXKJ

– Aristotle, (350 BC). Poetics, Translated by Stephen Halliwell. Chapel Hill: University of North Carolina Pres, 987. Pg 34.

– Baudrillard, J. (1981). Simulacra and Simulation. Translated by Sheila Faria Glaser, University of Michigan Press (1995), ISBN: 0472065211

– Bright, R. (2000). ‘Uncertain Entanglements’ in Sian Ede (red.), Strange and Charmed Science and the contemporary Visual Arts, Calouste Gulbenkian Foundation, London 2000, p.120-143

– Debord, Guy (1967). Society of the Spectacle, Zone Books, 1995, ISBN 0942299795

– Dennett, D. (1991). Consciousness Explained. Boston: Little, Brown, and Co.

– Goodman M., Tagle D.A., Fitch D.H., Bailey W., Czelusniak J., Koop B.F., Benson P., Slightom J.L. (1990) Primate evolution at the DNA level and a classification of hominoids. Journal of molecular evolution 1990;30(3):260-6.

– Gregory, R L. (1998) Eye and brain: The psychology of seeing (5th Edition). Oxford University Press.

– Hedges, S. B. (2000) Human evolution: A start for population genomics, Nature 408, 652-653 (7 December 2000)

– Heeks, R. (2008). Current Analysis and Future Research Agenda on “Gold Farming”: Real-World Production in Developing Countries for the Virtual Economies of Online Games. Working Paper Series, Development Informatics Group, University of Manchester.

– Hoffman, D.D. (1998). Visual Intelligence: How We Create What We See. New York: W.W. Norton and Co.

– Lakoff, G. & Johnson, M. (1980). Metaphors we live by, The University of Chicago Press, Chicago and London, 1980.

– Lumière, Auguste & Lumière, Louis. (1895). L’Arrivée d’un train à la Ciotat

– Mensvoort van, K. & Duyvenbode van, M. (2001). Het Bos ruikt naar Shampoo, Documentary. VPRO Television, April 2001.

– Mirzoeff, N., (1999). An introduction to visual culture, Routledge, London, ISBN: 0415158761.

– Nevejan, C. (2007) Presence and the Design of Trust. PhD. Dissertation, University of Amsterdam, 2007.

– Oosterling, H. (2000). Radicale Middelmatigheid (‘Radical Mediacrity’), uitgeverij Boom, Amsterdam. P23. ISBN:9053526218

– Oosterling, H. (2003). Act your actuality, in Gerritzen, M. et al. 2004. Visual Power: News. Gingko Press, Corte Madera, CA, 2004, ISBN: 9063690568.

– Piaget, J. (1997). Jean Piaget: Selected Works. Routledge, 1997, ISBN 9780415168892

– Plato (360 B.C.) The Republic, Translated by Robin Waterfield, Oxford University Press, USA

– Searl, John (1995). The Construction of Social Reality, Free Press (1997), ISBN: 0684831791.

– Winkel van, C. (2006) Het primaat van de zichtbaarheid, NAi Uitgevers, 2006, ISBN 90-5662-424-5

– Ijsselsteijn, W. A. (2002). Elements of a multi-level theory of presence: Phenomenology, mental processing and neural correlates. In Proceedings of PRESENCE 2002 (pp. 245-259). Porto, Portugal.