Based on two ethnographic studies of technology–driven innovation in music education and judicial practice, in this paper I investigate the nature and meaning of mediation as a primary aspect of our way of experiencing and understanding reality. I explore what happens in an established domain of practice when the introduction of new technologies, such as the computer and video recording, requires practitioners to work with a new medium for carrying out their practices. In spite of the apparent distance of the two practical domains, music and the judicial, the two cases point to surprisingly similar phenomena affecting the nature of objects, the relationship between objects and their representations, and the perceptual and practical skills of the practitioners. The paper shows to what extent a practice is embedded in the medium and discusses the coping strategies that musicians and judges enact in order to make sense of and master the new media, and to reweave the ripped fabric of their practice.
Remediated practices: Mediation and medium specificity
First case: Making music in the digital
Second case: Using visual evidence in judicial decision making
Discussion: Making sense of the practice in the new media
The diffusion of new media in human practices brings multimediality and the idea of mediation to the fore of our attention. It reveals the extent to which our practices are mediated by a web of objects, tools and representations whose features and functionalities are specifically associated with the media in which they are embedded. Multimediality has become a central issue across a wide range of practical and professional domains, and the diffusion and place of ‘new media’ in human practices have become an object of scholarly attention (Bolter and Grusin, 1999; Hayles, 2002; Manovich, 2001). When the new media replace or mix with traditional ones, a long established practice may be perturbed or even disrupted by the discontinuity. Different modes of integration must be sought between pre–existing and new objects, routines and representations in order to re–establish the practice’s ecological balance and the smooth flow of effective and unproblematic activity.
The idea of mediation, as I use it here, means that all human activities are dependent, although in different degrees, upon one or more media. Mediation can be material, that is, supported by material tools or by a material substratum, and semiotic, that is, carried by signs and semiotic conventions — an important distinction that I will refer to in the following and that can be traced back to the works of Lev Vygotsky (1986; 1978). Media shape our thoughts and practices (Hayles, 2002; Kittler, 1997; MacLuhan, 1994; Sturken and Cartwright, 2001), but they do it in ways that tend to slip out of our explicit awareness as we become increasingly familiar with them. When we engage in our everyday activities, the media in which we act and think tend to become part of the unquestioned background of our skills and representations — that is, they support what we know how to do (our effectiveness and sense of mastery) and what we believe there is (our identity and sense of reality). We feel at home with them. In different terms, media tend to become part of the functional and institutional circuitry shaping and supporting our sense–making capabilities, our collective intelligence, and our practical transactions and engagements with a variety of objects, tools, routines and representations. In this perspective, media are the modes of articulation with respect to which material objects, relations or ideas are manifested, materialized, and mediated in a domain of practice . Basically, a medium is what brings objects and relations to existence or else to concealment, and a change of the medium may either bring forth or annihilate, and always calls into question, specific features of our experience of reality (Chandler, 1996; Kallinikos, 2009). It is for these reasons that media are essential ingredients of our practical integration with the world .
In this paper I investigate the nature and meaning of mediation as a primary aspect of our way of engaging and understanding reality. I do it by exploring what happens in an established domain of practice when the introduction of a new technology requires practitioners to migrate to a new medium for doing work and accomplishing a task. I argue that the constitutive influence of mediation for the practice can be best appreciated in situations where highly medium–specific practices must migrate to a different medium. My inquiry therefore starts from the observation that many established practices are medium–specific, that is, the relevant objects, tools, routines, representation and skills that constitute the practice gain their shape, structure and meaning depending on the medium in which they have been formed and live, and will not be easily changed or re–shaped when they migrate to a different medium (Brown and Duguid, 1996; Hayles, 2002; Maras and Sutton, 2000). They are, so to speak, both ‘hosted’ by and ‘hostages’ to the medium. Consider health care for example, where the work routines of doctors, nurses, clerical staff and the entire organization of the hospital have traditionally been mediated by the paper–based patient record (Berg, 2000); or else, consider the practice of architectural design, where architects think and act with pencil and sketchpad, and eventually with the scaled cardboard models they make (Schön, 1983).
To varying extents, in many practical settings medium specificity may render migration across media problematic by incurring high learning and transformation costs for individuals and organizations. Established abilities and representations are not smoothly transposable across media and even simple work routines might be carried out with difficulty, at least in the initial phase of the transition (Dourish, 1993; Heath and Luff, 1992). In switching to a new medium, practitioners must engage in activities of redesign in order to cope with ambiguity and make sense of the unfamiliar medium. More specifically, they must restructure their relationships with the relevant objects and tools, routines and representations, so that the practice can be appropriately ‘hosted’ within the new medium. In other words, switching medium calls into question the nature of the familiar tools of the trade in a practice. At the extreme, the new medium may push the boundaries of a profession or a practice to the point that it becomes difficult to tell whether one is still working within an established domain of practice or else is doing something different. This makes the encounter with new media always a complex event, where the inherent properties of the media interact with the structure of the practice in ways that are difficult to be assessed.
The argument developed here is based on findings from in–depth studies of two major projects in two different settings: a) the development of a new computer music system and software for undergraduate education at a major research and academic institution in the U.S., and b) the adoption of video recording (VR) technology in the hearings and the practice of the criminal trial in Italy. Each case study is based on extensive fieldwork conducted for a period of two years through participant observation, interviews, ethnographic techniques and facilitating interventions. In the computer music project the main research site was the computer music laboratory where a music teacher and a programmer developed and tested different versions of the software; in the VR case qualitative data were collected by closely tracking the judges’ process of adoption of the VR in six criminal courts. Both studies had the character of practical learning experiments jointly conducted by the observers and the actors. In the first case doing computer music entailed learning to handle new objects, tools, procedures and relationships in a digital medium, and ultimately mastering a software program for making musical structures. In the second case judges and other actors were required to develop an ability to swiftly use the VR tapes of the hearing as valid legal documents for judicial decision making, thus integrating visual data into their traditional paper–based legal procedures.
At a first look one cannot possibly think of two more distant domains of practice than the musical and the judiciary. However, when we come to examine the phenomena generated by the change of the medium, the domains show surprising similarities. Although the media involved are different (digital and visual), both cases instantiate a ‘displacement’ generated in the practice by the intrusion of the new medium. Besides, although the semiotic aspect of the mediation is in focus in the computer music case and, on the contrary, the material aspect of it is dominant in the adoption of videorecording, both cases lead to similar questions around the nexus between the practice and the mediation. In both cases the practitioners are compelled to question their practice by reflecting upon the nature and meaning of the familiar objects and representations of the practice that they take for granted. Thus, their questioning concerns neither the effectiveness of the instruments nor the level of performance, not even just the epistemic content of the activity, but the ontology itself of the domain of practice, that is, what a practice is made of. In the following I will position my argument in the current literature and then I will use the cases to explore and articulate it in greater detail and depth.
Remediated practices: Mediation and medium specificity
The idea that practices are mediated activities seems so obvious that researchers have not bothered to pay much attention to it, or to ask what it means for a practice to be ‘mediated’. Also, the empirical observation that certain practices may become medium–specific has not led fully to appreciate and assess the implications of medium specificity for practical knowledge and for innovation. Yet, an early formulation of the importance of the idea of mediation for practice can be traced back to the path–breaking work of Lev Vygotsky on the development of human capabilities for thinking, speaking and acting (Vygotsky, 1986; 1978). Vygotsky shows how human capabilities are developed through the mediating function of materially–embedded artifacts like tools and signs. The use of tools and signs entails some kind of mediated activity leading to specific ways of mastering the external environment and structuring human cognitive processes. Thinking and acting are therefore regarded by Vygotsky as activities deeply entangled with a web of material and symbolic artifacts that take part in the development of practical skills. Vygotsky’s notion of mediation can be profitably extended from the field of developmental psychology to the study of practice and practical knowledge (Lanzara, 2009; Miettinen, 2009; Miettinen, et al., 2009) .
Based on Vygotsky’s analysis it is useful for my purposes to distinguish between the material and the semiotic (or symbolic) mediation of a practice: material or tool–based mediation takes place through the objects and tools of the activity and through the material milieu or substratum which supports the activity itself; semiotic or sign–based mediation is enacted by the semiotic conventions or symbolic means which underlie the representations, codes and languages that help perform the activity and convey the meaning (Kallinikos, 2003). In a domain of practice material and semiotic mediation play a relevant role, and they often come together. Both kinds of mediations have implications for each another: on the one hand different material tools or milieu may influence the semiotic conventions and the kind of representations that are possible or desirable; on the other hand the semiotic mediation ‘fixes’ the specific entities and units of description that are then held as the stable and relevant ‘objects’ of the domain.
Although all practices are dependent, in different degrees, upon one or multiple media, the degree of medium specificity of a practice may vary with the nature of the medium and the history and contents of the practice. In almost all practices there is always a ‘primitive’ or ‘root’ medium influencing the adoption and use of other media (for instance, clay, paper, video, or digital) (Plowman, 1994). The more deeply embedded is the practice in a specific medium, the greater the amount of restructuring involved in the migration to a different medium is likely to be. The issue of medium specificity has been raised mostly by scholars working in literary analysis and media theory (Bolter and Grusin, 1999; Hayles, 2002; Kittler, 1997; Kittler, 1999; McLuhan, 1964). “Medium specificity theories generally concern themselves with the idea that different media have ‘essential’ and unique characteristics that form the basis of how they can and should be used.”  But in spite of the spread of new media and technologies that require learning to work in multiple media the practical and theoretical implications of medium–specificity have often gone unnoticed in the practice literature. Indeed, the notions of medium and mediation are inherently ambiguous and not easy to grasp, because media tend to become transparent and almost invisible with use or habit. In its simplest, almost tautological definition, a medium is that which mediates between two (or more) elements or domains, that is, that which makes a ‘contact’, association or transaction possible between them. In the current transmission model of information, a medium is defined as a means of transmitting a message, and is essentially conceived as the material carrier of the message. Alternatively, in the semiotic model of communication the medium mainly refers to the templates for transmission, based on social, linguistic or textual conventions. In this sense a medium is rather the format or genre of communication by which the message or whatever information is structured, like the printed book, the newspaper, the talk show, business letter or e–mail (see, for example, Orlikowski and Yates, 1994). However, as we know from McLuhan (1964), a medium is not simply a neutral carrier or channel of things known, said or seen (represented), but actively shapes the informational content that it carries and the meaning that it conveys. In other words the act of mediation does not simply bring different elements into a relation, hence it is not just a neutral inter–mediaire, but actively participates into the making of the relation and of the activity, albeit in subtle, inconspicuous ways. According to Hayles (2002), one can only ‘do’, ‘think’ and ‘say’ what the medium allows one to ‘do’, ‘think’ and ‘say’. In this perspective, the medium is an active structure, not an inert substratum that is given once and for all. It actively shapes the entities and relations embedded in it, supporting and structuring specific modes of perception and action. It has an impact on selectivity and memory, that is, on the kind of cognitive processing that a subject is likely to perform (Chandler, 1996). Somehow a medium makes specific frames available for labeling and classifying objects and for recalling events and stories. When migrating across different media, objects, relations and representations do not simply ‘transit’ from the one to the other unaffected, but are ‘re–mediated’ by the medium (Bolter and Grusin, 1999). What happens in the re–mediation, then, is not a simple transition, nor a replacement, but the merging of two or more separate media giving birth to a hybrid in which the content is re–used, recycled and given different meanings (Bolter, 2001). Therefore the appearance of a new medium in a specific domain of practice, whatever conceptualization one may prefer to account for the phenomenon, is always a complex and dynamic event, involving a blend of discontinuity and continuity, disruption as well as construction.
When the medium changes, a critical consequence for practice is that familiar objects and work tools may lose their functional and ontological stability, inasmuch as such stability also depends on the stability of their relations with the background (Brown and Duguid, 1996). The combination of materials, representations and media undergoes a ‘re–setting’, and problems of recognition, representation and performance may emerge. Ambiguity grows, and objects and tools suddenly become strange and elusive, showing unexpected behaviors that make smooth, unproblematic handling difficult. Consequently, the agents’ practical and sensemaking abilities are affected, because artifacts, tools, routines, and the medium itself are partly experienced in terms of the activities they immediately lead to perform (Kelly, 2005). Similarly, the amount of interpretive flexibility available to actors is influenced by the medium (Collins and Pinch, 1982). When an agent feels that she is having problems in perceiving and handling what she thought to be a ‘known’ object or situation, she experiences a kind of disability: some moves and routines that were performed in the previous medium cannot be performed or must be differently performed in the new medium. In the shift produced by the new medium, the boundary between tacit and explicit knowledge also shifts and becomes problematic, because tacit knowledge is formed in relation to specific tools, materials and situations of action (Cook and Brown, 1999; Polanyi, 1967; Sternberg and Horwath, 1999). Practitioners must then engage in reshaping their ways of practicing in order to recover the ‘felt path’ and reestablish smooth performance. As they do so, they may also discover that the new medium brings forth new objects and tools that support new capabilities, novel ways of working and knowing, and multiple organizing processes. In turn, the adoption of new tools and means for work may lead to new forms of mediation (Vygotsky, 1978). What I am pursuing in this paper are precisely those fleeting moments of rupture and re–design that practitioners experience when practices migrate across different media and undergo a remediation. Thus, rather than assessing how smart or how fast agents are at achieving a high level of performance in a given medium, I will focus here on the forms of understanding and practical knowledge that develop when agents must cope with the new objects and relationships that are brought to existence by the new medium. Therefore, not the knowledge/performance nexus within a given medium is of interest here, in a purely instrumental or managerial perspective, but, perhaps more in the spirit of an ontological and epistemological investigation, the phenomena and problems that arise when the knowledge/performance nexus is ‘carried over’ and restructured across different media.
First case: Making music in the digital
My first illustration deals with the development of a computer music system and language in a music school of a major U.S. academic institution to help students experiment with musical structures and expand their musical understanding. The idea was to develop a system that would allow students to compose and play music using the computer as a medium. At the outset the two system developers — a musician and a programmer — spent several weeks in the computer music lab of the school designing an appropriate interface between two distinct domains of expertise: music and the computer. Doing computer music involves learning to work in a digitally–based medium by using a coding language and inputs that the computer can understand. The developers’ attempt was to develop software tools and procedures for making music in the new medium. Depending on the developers’ design choices, users would be given different entities, relations and tools to play with, and would have access to the music domain and to relevant musical objects at different levels of aggregation. As the musician said, “I want to give them a world of things to play with, think with, and act upon.” Different types of interface and intersect between the two domains would have different educational implications for music teachers and students. As the developers jointly searched for an appropriate ‘entry point’ to the music world, they tried out different kinds of technical and functional intersects between the computer and the music domain, giving shape to specific objects that are delivered to the users. The basic problem they faced was how to represent music in the digital (that is, as a software program) so that it could be inputted to and processed by the computer and what the available or new software could do to that purpose. Remediation of the practice of musical composition takes place here primarily by changing the semiotic conventions, that is, the language for describing musical structures.
Splitting the note apart
In the traditional music environment musical material is represented through a standard symbol system based on conventional staff notation. A note encapsulates two basic entities — the pitch and the duration — in one unit symbol that comes as a whole. Tunes are composed by writing notes on the score and positioning them on the pentagram, then played with an instrument by reading the score. The ‘stuff’ of music is thought, made and acted upon through the mediation of the notational language. This is how ‘sound’ is structured, represented, and ultimately turned into ‘music’.
But what happens to pitches and durations when they migrate to the digital medium? Which new features does the medium bring in and which pre–existing features does it obliterate? What new things and actions can be done in the new medium that could not be done in the conventional one? And, conversely, what is lost?
The first thing the developers discovered when they set out to make music in the new medium was that musical objects and relations do not quite ‘migrate’ across media untouched. Rather, they are re–shaped and gain different properties, to the point that the way one thinks about the structure of the music domain can be dramatically affected. The computer music system exploits the powerful properties of the LOGO programming language, which can process independent data lists and inputs them to the computer. Within the digital medium, LOGO allows a disconnection between pitches and durations, breaking the wholeness of the note. This is the musician’s account of her design ‘invention’:
|“The system requires two separate parameters to define pitch and duration. In standard music you have only one symbol — the note — that contains both the pitch and the duration. The students get a very clear idea that any note has these two parameters: pitch and duration. Something interesting happens conceptually when you rip apart the two parameters that are locked into one symbol. You begin to realize that you can manipulate them separately, which you can’t do when you’ve got the score sitting there. Well, you could do it, but you don’t think of doing it.”|
Unlocking pitch and duration from the staff notation and treating them separately as independent lists of features and values allows one to explore how they interact and to do things that would be hard to do with a musical instrument, but are very easy to do with the computer:
|“Think first about just the pitches and then about just the durations, and then hear them together. The idea of having a list of pitches and a list of durations was an invention. It changes the way you think about melody … .You see, when I saw the LOGO language I realized right away that it was perfect for music, because what you want to do is manipulate lists of features and lists of values for those features. That is exactly the way you want to think musically … .It allows you to do things that computers do all the time, like keeping the pitches the same and changing the durations. But once you do that you begin to realize how those things interact … . It allows people with no music background to play with the materials of music in a way that a composer does.” (musician)|
The developers discovered that the computer language allowed for a different treatment of musical objects in the new medium. As they were transposed to the digital medium, objects and properties were also transformed in ways that liberated new features of music itself. New things could be done with those objects and new representations emerged. A whole range of new activities for making music took shape: first inputting numerical data, then running the procedures for manipulating and editing structures and for making tunes, finally having the computer play the music through an internal MIDI synthesizer .
When switching to the computer–based medium, the practice of making music becomes a somewhat different thing. Computation penetrates into the domain of music. Music can be composed, manipulated and played only through the mediation of a software program. The musical material must be codified in data structures that can be inputted to the computer by punching them on the keyboard and clicking the mouse . Whole musical structures and data can then be stored in the computer memory, retrieved, and variously manipulated when the musician wants to make some tune or play around with musical variations.
However, in experimenting with Music LOGO the developers also realized that the list structures and procedures were quite distant from the familiar input/output representations of music associated to the notational system. Therefore they set out to develop system features that allowed the inputting of data in various formats: standard staff notation, alphanumerical, and graphical/pictorial (Bamberger and diSessa, 2003). They wanted to create a user–friendly software interface that would allow users to do direct inputting and perception–driven manipulation and editing of musical structures, that is, a language that would make computer music as close as possible to the actual practice of music and to the users’ sensory, perceptual experience of musical material. What the musician was aiming at in her development efforts was an immediate physical way of inputting the computer, allowing for a swifter connection between perception and symbolic description and for more flexible sound production routines. The new medium did not necessarily preclude the possibility of inputting musical motives using the standard staff notation, but also made other forms of representation possible, as the software programmer pointed out:
|“The input and output representations need not be confined to list structures. An important aspect of the system is its capability for multiple representations. One should be able to input musical motives in conventional staff notation form, for example, and then be able to manipulate the resulting data structure utilizing the procedural capabilities of the system. The list structure created from this staff notation input will then be available to the user for editing and /or viewing purposes. This feature will be of great use to those users who are used to composing in conventional notation only. Likewise, inputting data in various graphical forms, that can also be manipulated procedurally, should be implemented. Furthermore, higher level musical structures should have graphical representations as well … .” (my emphasis in italics)|
In the musician’s own words:
|“Instead of typing numbers in with the keyboard I want to input the computer through a gesture, using the mouse as a vehicle — a gesture which will be seen on the screen — and that gesture will turn into sound.” (my emphasis in italics)|
The design strategy here consists first in “going down” into the computer guts and the machine language, so as to turn music into digital data in order to make it inputable to and manipulable by the computer, and then in “climbing back” up the ladder from the binary code to an iconic code that would provide a more sensory–based, intuitively accessible representation and composition of musical structures. In the process of experimenting with musical–digital interfaces at different levels, new software entities are developed (windows, icons, waveform graphics), and with them also new mappings between actions, events and objects, which the musician must learn how to work with. At the same time the developers learn that music can be perceived in multiple sensory modalities and represented in multiple representations, which are encountered at different steps of the ladder (Bamberger and diSessa, 2003). Digital sound may well be a string of symbols in the computer but at higher levels gives birth to a range of objects which lend themselves to phenomenological investigation.
Composition or programming?
When the prototype version of the system left the Music Lab and was presented to the Music Faculty in a demo, so that it could be tested as an educational and compositional tool, it raised a lot of questions:
|“Is this real music or is it just playing with numbers? Is composition being turned into programming? What do the students get out of this thing? Are they really getting involved with music?” (music teacher, instructor of the course Introduction to music)|
Some teachers questioned the educational value of the system and regarded its integration in the music curriculum as problematic. They were wary that basic ‘performing’ class activities like listening and singing would be partly replaced by programming and procedural analysis, and that the new tool would “turn music into numbers.” What they questioned mostly were the non–standard representations and the procedural dimension that the digital medium disclosed, which they believed to be at odds with the essence of music and the overall purpose of music education. But other teachers, who were more composition–oriented, quickly picked up the system and tried out smooth compositional routines. They directly engaged the system as a compositional medium for experimenting with a variety of musical elements and immediately went into musical matters such as coherence, structure, symmetry, periodicity, metrics, groupings, accentuation, etc. Particularly, they were puzzled and curious about the ‘strange objects’ and the multiple representations of musical materials that the system and the new medium made possible. As a whole they were impressed by the potential of the tool for creative uses and for providing the students with hands–on experience of musical materials.
Although teachers had different reactions to the novelties that came with the new medium, they all faced the question of how to make sense of the new objects, procedures and representations of musical materials that came with the digital medium (Lanzara, 1991). The displacement from their familiar, medium–specific world of music practice generated a temporary loss of mastery and meaning . As they switched to the new medium and engaged in making music, they had to re–learn their whereabouts and some elementary routines. A large part of their learning and sense–making consisted in mapping out objects, procedures, activities and representations onto one another across the different media. It meant learning how to work with multiple representations. In the process, as the music teachers learned to think and do things within the new medium, they also (re)learned things and features about the traditional practice of making music that they were not aware of or had forgotten about. Indeed, as Katherine Hayles has convincingly shown, the new medium does not replace or incorporate the previous one, but ambivalently ‘reverberates’ on it by making it richer with new meanings on the one hand, and, on the other hand, by erasing pre–existing features and possibilities (Hayles, 2002).
Second case: Using visual evidence in judicial decision making
My second illustration originates in an in–depth study of the process of experimenting video technology for documentary purposes in Italy’s criminal courts (Lanzara and Patriotta, 2001). When a prototype of the VCR system was introduced and tested in the courtrooms, judges and lawyers stumbled upon a new object: the videotapes containing the film of the hearing. A video is a different kind of object than the transcript: the transcript ‘tells’ or ‘reports’ an event, the video ‘shows’ it. The representation of the event happens to be inscribed into different media: paper transcripts and visual images. The videotapes immediately became a practical and interpretive problem for the judges. Their established practices and representations of the trial were perturbed by the sudden appearance of an apparently ‘alien’ object. Their questions were: What kind of object is the videotape, and what it can offer that the paper transcript does not offer? Can we use the video and the data in it as formal documents for judicial decision making? How can we integrate the video in our practice and in the courtroom activity system? These questions originated directly from the different kind of material mediation that the video brings into the practice.
In the long history of the legal profession the trial record in the format of a paper transcript has been a key artifact for reducing the ambiguity inherent in a legal dispute. Judges have developed medium–specific practical routines and skills, which are coupled with paper–embodied artifacts, such as minutes, files, folders, archives, not to mention the writing tools. Paper as a medium and the transcripts inscribed in it subtly influence the work practices, the reasoning strategies, the representations of the actors, and even the courtroom layout and the back office organizational routines. The switch from paper to visual caused a cognitive displacement with the judges, who were pushed to reflect on the nature and the meaning of the evidence produced in the trial and on the modalities through which visual evidence can be used to support judicial decisions. Some of the judges went as far as to question basic, taken for granted assumptions underlying the legal profession. Let me briefly illustrate the most critical aspects associated to the visual medium, as they have been experienced by the judges.
Access, selection and mapping of visual evidence
First of all, the new medium changes the relationships that judges, lawyers and legal clerks have to the records of the trial. It affects how data potentially relevant for judicial use are accessed, selected and mapped. While paper records are easily accessible and immediately inspectable, videotapes need technological equipment to be run, scrolled and inspected. In other words a paper transcript allows random access, easier selection and focus upon specific points and, overall, more ‘reading’ flexibility than does the videotape . In the new medium familiar paper–specific acts like using pencils or colour markers, underlining, highlighting, writing notes on the edges, using ‘post–it’ stickers to signal pages, make references and quicken searches cannot be done. On videotapes, search, retrieval and inspection of salient snapshot information (punctual events, facts, data), such as a crucial deposition, must be performed by using pointer, index, and go to … devices. Actors in the trial felt that, when using the videotapes, they could not concretely “fix the relevant points”, as one of them said. As a consequence, their sense of personal mastery was affected. A judge remarked that he “could never ground his decision on something so elusive, so evanescent” as the VCR. Another judge said that he could “not really grasp the logic of the tool” and could not have a “firm hold” of the data and the task. Similarly, standard routine operations like copying, classifying and filing the video records became “different things” in the new medium, requiring specific technologies and criteria, and appropriate office spaces and scaffolds.
Secondly, the VCR brings about, in the visual medium, a new type of representation of events. Videotapes inscribe not only more data than paper transcripts, but most crucially data of a different kind. Events (and the connections between them) reproduced in a visual medium are different, and differently articulated, from events and connections reproduced in a written form. They induce transactions and engagements of a different kind, too. Therefore a one–to–one mapping of events and relationships across the two media cannot be easily done. Particularly, the new visual medium induces a different perception of what completeness and accuracy of data mean. The judges were thus compelled to reflect on the traditional criteria for assessing and using evidence, which were anchored to the paper medium. For instance, some judges argued that being able to look at a witness’s behavior on a monitor led them to a different perception of the witness’s reliability, thus also leading them to interpret the witness’s statements in a different way.
If criteria for establishing evidence and truth are medium–sensitive, then the core task and competencies of the judges are affected when they use the videotapes. The videotape might lead to discover or construct different ‘as–therefore’ connections than the written transcripts would do and, consequently, to higher ambiguity as to the relevant ‘if–then’ precepts that should be invoked to settle a controversy . All the parties in the trial — lawyers, prosecutors, judges — have to redesign their strategies for dealing with alternative versions of reality. In other words, they have to learn to build, work with and compare multiple representations of the data, each one of which may convey a different picture of the event and the judicial case .
Finally, the VCR affects the judges’ selectivity and consequently their relationship with their own memory. Judges are used to anchor their memory to paper transcripts. However, different media support specific ways of directing selective attention and retention. Observation of judges’ behavior significantly revealed that when watching the videotape they tended to perceive a stream of events in a continuum, while when reading the transcripts they tended to separate one event from the next. As a result, in the paper medium punctuated search of distinct episodes through focusing, chunking, parsing and grouping of data was more likely and easier, while the video better conveyed the experience and eventually the emotion of re–living the story of the trial. Selectivity was also affected by the different perception of time that the judges experienced in either medium. They complained that reviewing the hearing on the video was extremely time consuming, as such activity tends to be temporally co–extensive with the real event. Indeed, while judges can be extremely selective and effective in browsing and studying hundreds of written pages packaged in thick reports, when they switch to the visual medium they experience a loss of focus and concentration and tend to be carried away with the flow of the story. In other words, in the experience of the judges, the visual medium lacked the properties of reducibility, compressibility and selective readability of the data that the paper transcript allowed.
The legal use of visual evidence: The problem of non–verbal behavior
In their efforts to make sense of the new tool, the most critical issue raised by the judges was the question whether the videotapes should have the same juridical status and legal validity of the paper transcripts, and whether they should or not be adopted as the official, complete record of the trial. As we have already mentioned, the videotapes offer access to immediately rehearsable and inspectable inscriptions of non–verbal behavior, which are not stored in the paper transcripts. Indeed, access to non–verbal cues modifies the judges’ relationship to the reproduction of the trial proceedings, and consequently to the process of constructing the proof. Non–verbal data was a source of puzzlement for the judges. Controversial views of the normative conformity and the practical usability of the VCR system were aired. Some judges were in favor of it:
“There can be no doubt about the usefulness of the VCR, because it records not only the voice, but also the gestures of the actors, the movements, the signals and hints that they show and that cannot be reported in the written proceedings … It is important that I can see these signs again, these ways of behaving, of acting, because it can generate meanings that are often not clearly interpretable.” (judge 1)
“Video recording is the only system that allows for a complete documentation and assessment of the proof.” (judge 2)
Others, instead, considered that nonverbal cues and video recording in general were incompatible with the formal correctedness of the legal procedure:
“It is quite difficult to refer to a behavior of the defendant or the witness that is not what she tells.” (judge 3)
“The personal perception that a judge can have of non–verbal cues visible in the videotape, such as the defendant’s body gestures or facial expressions, are not univocal and, in any case, there is no way they can be used to officially support a verdict.” (judge 4)
These divergent stances were occasioned by the peculiar capability of the new medium to capture and fix non–verbal cues, allowing for potentially endless repetitions that make them inspectable and disputable. The controversy goes right to the core of the judicial profession. The most sensitive judges, challenged by the presence of the new tool, responded reflecting on issues of objectivity and subjectivity and on the medium–specific and relativistic nature of their evaluation criteria. Some of them went as far as to explicitly acknowledge an incoherence buried in their practice: on the one hand, according to the code of criminal procedure and to the accepted practice, only verbal, declarative data released by processual actors and reported by the paper transcript should be used to support a judicial decision, excluding all subjective appreciation of nonverbal behavior as ‘unlawful’; on the other hand the new medium revealed that the judges tacitly used and valued non–verbal cues in their judgments, even though they would never explicitly refer to them as ‘proofs’ (or as evidence for a proof) in the official (written) text of a judicial verdict. Exposed to their own inconsistencies, the judges enacted hybrid behavioral patterns and tentative lines of reasoning that reflected their own efforts at integrating the new medium within traditional legal procedure.
Discussion: Making sense of the practice in the new media
The two case studies provide useful materials to discuss the nexus between practice and mediation. In spite of their obvious differences, the cases show a similar pattern associated with the introduction of a new medium, so that some bridges can be thrown across them in order to capture the common aspects and gain a better understanding of the phenomena observed. The new medium invites the actors to question and reassess the nature of the objects, representations, and practices. Indeed, it has been remarked that “the introduction of a new technology always seems to provoke thoughtfulness, reflection, and self–examination in the culture seeking to absorb it.”  In our case studies the encounter with the new medium triggers an experimental and reflective activity that goes right to the core of the practice. Basic forms of practical knowledge and expertise and, more in general, what really counts in a practice, are called into question. Interestingly enough, such inquiry often proceeds by ‘practicing out’ and testing incoming thoughts and alternative arrangements. Puzzles lead to probes, which in turn lead to new questions. The implicit questioning underlying the actors’ experimental activity can be framed in parallel as in the following table:
|Case 1: Making music in the digital||Case 2: Using visual evidence in judicial decision making|
|What is a tune made of?||What kind of document is the videotape?|
|What is a musical structure?||What is a proof?|
|How should we represent the ‘stuff’ of music?||How should we represent the ‘facts’ of the trial?|
|What is musical composition in the digital medium?||How can we use visual data to make a judicial decision?|
It should be noticed that in the first case inquiry begins with questions about the modes of representation (the musical notation), that in turn lead to questions about the nature of musical objects to be represented and about their relationship to the representational system, while in the second case inquiry begins with questions about the material medium, that in turn lead to questions about the appropriate representations of legal facts. The questioning reveals a complex relation between the inherent properties of the media and the ways they are experienced by the actors. In the following I will discuss some of the problems the practitioners face as they learn to deal with the new objects and representations that the remediation of the practice brings about.
Coping with strangeness
Nothing could be more illustrative of John Dewey’s observation that “the object is that which objects” than the situation triggered by the sudden appearance of a new medium in an established domain of practice. Switching medium produces a discontinuity in the smooth flow of familiar routines and a displacement in the practitioners’ perceptions and understandings of what they do and how they do it. Practitioners encounter new objects and tools that look strange and elusive, or rather, deceptively familiar. The medium somehow ‘channels’ attention to specific features and relations that are perceived instead of others. In the early stages practitioners find it awkward to establish smooth practical transactions with the new things that come along with the new medium. Habitual actions do not produce the expected outcomes, as if objects and media manifested logics and a life of their own that shrewdly resist or deflect human purpose. Familiar routines do not show to be effective when they are transposed to the new medium, and need to be adjusted or redesigned, while new routines are not available as yet. Tools and implements do not come handy and are difficult to maneuver, like the non standard notation in computer music or the videotape in judicial work. Ordinary objects become recalcitrant and obtrusive. When they are handled or acted upon, at the outset they respond in ‘weird’ ways. In Lorraine Daston’s (2004) evocative expression, things that have long been ‘silent’ begin to ‘talk.’ Indeed they seem to ‘object’, by failing to support human agency, purpose and sense–making. Strangeness suddenly arises, together with the urge to make sense of it .
The new medium opens up a window for novel things to happen in a new space of possibilities, revealing features of the domain that were not visible, or not easily accessible, or rather not paid attention to, in the traditional medium. It exposes what actors take for granted in their everyday dealings, that is, their ways of representing objects, together with their ways of entertaining practical transactions with them. If they want to gain familiarity and ease with the new objects and tools made available by the new medium, and make sense of them, practitioners must re-orient their sensory and perceptual maps.
The emergence of strangeness triggers perceptual and cognitive activity. The process through which the practitioners — both musicians and judges — learn to handle the new ‘things’ and gain familiarity with the medium takes place by means of local practical experiments. In the experiments the actors explore the nature and behavior of the new objects and tools and try to integrate them into their activity system: repositioning objects, adding or subtracting features, mapping relationships, identifying patterns, and exploring opportunities and constraints. For example, in the design of the computer music system the developers discover that two apparently distant domains — computers and music — may interface and interact at different level of aggregation of the relevant units. They realize that not all the features programmed in the software provide a smooth point of entry into musical matters for the users; not all the interactive interfaces between the music and the computer domain satisfy compositional or educational requirements; and not all non–standard notations tested by the developers turn out to be appropriate means of representations of musical materials.
In a similar fashion, the judges try to position the videotape within the pre–existing system of practical activities that constitutes the criminal trial. They must turn the video into a legal object, so that the evidence inscribed in a visual format can be legally relevant and usable. However, in the process of experimenting with it they discover that the video and the visual replicas put different kinds of requirements, which diverge from the usual paths and rules of the practice. In other words the video exerts a ‘pressure’ both upon the judges’ established routines of running the trial and upon their hitherto unquestioned views and premises of what counts (or doesn’t) as legal evidence.
In order to make sense of the new objects and draw effective cross–mappings between the different media, both the musicians and the judges engage in a design game which entails two opposing and complementary moves: first, they translate the traditional medium into the new one, and then, in reverse, they translate the new medium into the traditional one. Both mappings entail a great deal of transcoding and testing for the purpose of building compatibilities across media. Music or sound, converted to digital objects, must be compatible with and transcodable into computer files, but at the same time computer files and database structures must be formatted in ways compatible with features and requirements that are specific to the domain of music. Similarly, in the case of the judiciary, legal and judicial material (the events of the hearing), converted to visual objects, must be managed in ways that are compatible with visual rendition and perception, and, conversely, videos and visual replicas must be formatted and accessible in ways compatible with the specificity of the procedures, rules and traditions of judicial practice.
Remediating objects, skills and representations
As they proceed in their redesigning activities and in their efforts at cross–mapping between media, both musicians and judges engage the fundamental problem of representation. Confronted with multiple media, they experience the difficulty of fixing a non–ambiguous representation of their relevant experiential material: music or sound for the musicians, judicial facts for the judges. Indeed, the practitioners in the two cases face the basic task of discovering or reconstructing an appropriate representation or description of the domain so as to accommodate the new medium and the relevant objects and relationships it brings about. Building a representation involves an iconic transformation of reality through the material and semiotic mediation of a medium (symbol system, language, or material substratum). The representation zooms-in on some selected elements, which become the focus of attention, while others are ignored or pushed onto the background of perception and cognition. Both the musicians and the judges dramatically discover it when they are suddenly exposed to multiple and almost simultaneous representations of the stuff they deal with in their practice.
In switching medium new features are accrued to the relevant objects while others are deleted or minimized. The representations made possible by the new medium make things observable, which in the former medium were unobservable. For instance, in the computer music system the LISP–based representation makes it possible to explicitly describe and separately treat the basic component elements of a note — one could say the ‘primitives’ of musical material; similarly, in the VR system, videotapes expose data which are not inscribed in the paper transcripts or are of a different kind. The medium ‘confers’ new properties and meanings to the objects. The switch not only gives rise to different ‘observable entities’, but also affects the very idea of ‘observability’, because the perceptual segmentation of an object’s properties or of what is to be understood as an ‘event’ or ‘action’ is a different operation in the different media. As a consequence, ambiguity is generated and sensemaking becomes problematic, to the point that it is not clear anymore what an object (event, action) is, or at least whether in the migration from one medium to the other an object stays the same or changes in kind — becoming a sort of ‘mutable’ entity .
For instance, what is a tune, conceived as a digital object, and how should it be represented? Can a numerical sequence representing a tune be called a ‘musical object’? In the digital medium the musical material becomes a list of numerical data. A tune, then, is a combination of selected items from a database. It is an order given to a list. But does manipulating lists of numbers or lines of code amounts to composing musical structures? From our findings it appears that the practice of composition in the digital is experienced differently than in a traditional music environment. For once, the digital medium emphasizes the combinatorial variability of the musical materials. Digital composition amounts then to assembling together modular units and elements in various combinations to produce a single, seamless object. The tune so composed (or so computed) is therefore only an instance of a potentially infinite range of versions of a musical object. For sure, such potential for variability and the combinatorial dimension both belong to the essence of music and are by no means absent in a traditional music environment, as all musicians know. However, they tend to be implicit and not immediately obvious: the conventional notational language and the medium do not allow seeing them. The combinatorial space is laid bare by the new medium, which reveals features of music that are not self–evident in the traditional environment, because they are encapsulated and locked in a particular, medium–specific representation based on the conventional staff notation. In other words, in the standard musical notation and in traditional composition the database from which the tune is constructed tends to be implicit, while the actual tune is explicit and fixed (materialized) on the pentagram. In computer–based composition, instead, a sort of inversion happens: the database is fully explicit (the LISP structures), while the tune is just a virtual, mutable ‘version’ of a musical object. However, although it is perhaps perceived as a more unstable and volatile entity, it would be incorrect to say that the tune is real or material in the former instance while it is unreal or immaterial in the latter: in a phenomenological sense, both have a specific kind of materiality and ‘realness’ which is dependent on the specific features of the medium.
If we now move to the case of judicial decision making it appears that, although in a different medium and in a distant practice, judges struggle with similar problems and ask similar questions. They face the problem of integrating the visual data carried by the video into the practice. Integration is not immediate, because the video inscribes materials of potentially legal relevance in ways that differ from what the paper medium does. It replicates the event in ways that may change the relationship of the judges to the event, and consequently the meaning of the event — its function and significance in the legal procedure — are changed too. However, the video does more than just replicating events. It is also ‘productive’ of new events by making them visual mediation is indeed ‘productive’ of new features — non–verbal data — and these may become significant events in their own right, that need to be dealt with in the trial procedure. Non–verbal data do for sure belong to the essence and the drama of the hearing, but are effaced in the paper–based representation, becoming unavailable. Although they may occasionally and tacitly be recollected by the judges, they do not have the status of usable legal facts. As a judge said, they belong to the ‘tacit background’ of the verdict. But the video brings them to the fore and makes them available again. As the case study shows, this is an unsettling experience for the judges. The surfacing and observation of non–verbal data allowed by the visual medium have perceptual, cognitive and practical implications. First of all, the judges perceive the hearing as a continuous flow of events and actions when they look at its visual replica, therefore it becomes difficult for them to perceptually punctuate and identify specific, spot–like events or actions. In other words, they have the problem of segmenting the flow (‘chunking’) and positioning the events throughout the flow. Secondly, they must learn to go from the events perceived in the video to a skeletonization of facts that must be compatible with standard judicial procedures, and ultimately with the construction of the proof. This means becoming skilled at building new and richer representations of an event. Thirdly, they must adapt or redesign legal procedures capable of dealing with evidence, which is not just laid out in words writ, but in words spoken and images that come together simultaneously and in a perceptual whole.
To sum up, with the advent of the new media, the practice of their craft suddenly became problematic for the musicians and the judges, not because objects, representations and actions were temporarily misaligned (Vaast and Walsham, 2005), but rather because their nature and mutual relationships changed in the migration from the old to the new medium (and in their interaction). Therefore the displacement and the reflective questioning were caused not so much by dissonance and disconnection between objects, actions and representations (Vaast and Walsham, 2005) as by the sudden and simultaneous transposition of all of them to a new, and foreign, medium.
It is important to stress that in both cases the change of the medium has a revealing effect. It discloses and makes explicit what in the previous medium is implicit. Practitioners who engage with the new media are confronted with multiple representations and must learn to work with them. This may have a dramatic impact on the practitioners’ view of their own practice. Both the musicians and the judges sense that what they habitually take as the reality of their domain — what they know how to do and what they believe there is — is indeed constituted of ‘multiple realities’. When the music teachers try to map what they do and know within the standard medium into the digital medium, they discover the fictional character of their representations and the medium–specific character of their skills. The judges go precisely through the same experience when they confront the problem of matching the real events and their virtual replicas, and learn to make sense of the data in the visual medium: they discover that what they do is akin to constructing plots and story lines, not so different from what a playwright does. Knowledge is fictional and ‘fabricated’ or, in other words, fashioned by and within a medium. Through the mediation objects come into being, change and disappear, and so do our representations of them. Switching medium discloses the fictional element in knowledge making by making it evident that each one of the multiple representations tends to be built as a self–consistent and self–referential reality .
Self–referentiality of representations does not immediately surface in the non–problematic execution of standard routines (be they musical or judicial). When a practice is established and runs smoothly without being perturbed by alien events, representation is indeed there, but sits quietly in the background of experience. Media tend to become transparent with time and use. Think about making music or running a criminal trial with standard tools in a standard setting: everything — objects, routines, setting, people — seems to possess an aura of reality, the way things unquestionably are, marked by a sort of ontological closure and fixedness. But when the new tools and media come along then one discovers what in fact one already knows, or has known for a long time and perhaps forgotten: that objects are ‘made’ and that practical knowledge is heavily medium–dependent. The contours of things known, the boundaries and relations among them, need to be redesigned. The loops connecting objects, tools, routines, skills, minds and settings do not run smoothly and swiftly. Established closures do not hold anymore, objects and circuits ‘open up’ again and new closures need to be found. Actors are confronted with multiple, elusive ontologies.
In this paper I have tried to shed some light on the phenomena that emerge when new media are introduced and used into established professional and organizational settings. The appearance of a new medium in a domain of practice produces a perturbation in the complex ecology of agents and activities, objects and tools, uses and meanings that constitute the practice. Such unbalance leads to discontinuities in the smooth running of practical activities and to a temporary weakening of the sense–making abilities of the actors. The fabric of the practice is loosened or even ripped off, and needs to be rewoven. The medium is to be given a proper place, function and meaning within the practice, and in turn the standard objects and implements of the practice must be ‘remediated’ by the new medium. Remediation demands a perceptual and cognitive shift and the online redesign of routines. New mappings must be traced to connect the different media and the different ‘realities’ supported by the different media. To this purpose new representations and routines have to be constructed, new skills and a new configuration of the domain must be developed. The reweaving, though, cannot be done at once. Reestablishing the ecology requires the gradual accumulation of a repertoire of experiences with the new medium, or rather the skill and habit to repeatedly switch back and forth across different media and multiple representations, which can only be learned through time.
With all their limitations, the findings may lead to new insights about the nature and meaning of the objects and representations that we use in our practices, the practical knowledge that we develop, and the overall structure of a practical domain. They show the extent to which a practice is embedded in the medium, therefore is shaped by the ‘mediating’ and ‘formative’ influences of the medium. The findings also show how practical knowledge is tightly intertwined with medium-specific objects and representations, and how remediation, by modifying our perceptual and cognitive space, calls for different forms of coupling between objects, cognitions and skilled behavior. Finally, they help us to see more clearly that objects, routines, representations and practical knowledge are all linked together, loosely but pervasively, by multiple threads in practical settings. Such threads may well be subtle, inconspicuous and hardly visible to practitioners and researchers, being often hidden in the fold of the mediation, but they constitute the underlying texture of the fragile web of things and meanings that makes what we happen to call ‘reality’, and our knowledge of it.
About the author
Giovan Francesco Lanzara is Professor of Organization Studies and Political Science at the University of Bologna, Italy and since 2006 a visiting professor in the Information Systems and Innovation Group, Department of Management at the London School of Economics and Political Science.
E–mail: giovan [dot] lanzara [at] unibo [dot] it
The ideas developed in this paper have been extensively discussed in a number of occasions, notably at the Sociomateriality Workshops organized by Wanda Orlikowski and Susan Scott at the London School of Economics in June 2008 and 2009, and in a research seminar led by Barbara Czarniawska at the Goteborg Research Institute in January 2010. The paper owes much to enriching discussions with several colleagues, particularly Carsten Sorensen, Christian Licoppe, Jannis Kallinikos and Bonnie Nardi. I am especially grateful to Jannis and Bonnie for their very careful reading and useful comments on previous drafts of this article.
1. Sturken and Cartwright, 2001, pp. 169–170.
2.Medium, media and mediation are complex and still ambiguous concepts, and in spite of the many studies undertaken the ambiguity has not yet been fully cleared up in the literature. In this article I tend to use the terms metonymically, referring sometimes to the material substratum (paper, digital, video), sometimes to the technological artifact proper (computer, VR), or else to the semeiotic code (staff notation, software, text, image), where the one often implies the other. My choice of the term mostly depends on contextual and expressive requirements.
3. Particularly, Miettinen (2009) shows how Lev Vygotsky and John Dewey have developed parallel ideas on the notions of activity and artifact mediation, both feeding Cultural–Historical Activity Theory and a practice–oriented approach to social science and organizational analysis.
4. The expression “tools to think with” has been coined by Seymour Papert (1980).
5. MIDI is an acronym for Musical Instrument Digital Interface.
6. It is interesting to notice here that these new opportunities are created through a sort of ‘analytical reductionism’ which is inherent in computation. In this sense composition of music happens via the decomposition of a perceptual and representational unit into two items (pitch and duration). On this aspect of computation see Kallinikos (2009), who also called my attention to it. One might add, for sake of precision, that the entire interface through which a musician interacts with her materials is reshaped.
7. Using a term from Martin Heidegger’s hermeneutic phenomenology, one could also speak here of a loss of ‘equipmentality’ (see on this Dreyfus, 1991, chapter 4).
8. It should be reminded that in the Middle Ages the codex was heralded as a great improvement over the scroll precisely because it allowed random reading.
9. Clifford Geertz sees adjudication as “the back and forth movement between the ‘if–then’ idiom of general precept, however expressed, and the ‘as–therefore’ one of the concrete case, however argued.” (Geertz, 1983, p. 175)
10. Ron Burnett has explored a variety of interactive experiences that humans entertain with the new image–based technologies. He points out that so much intelligence is programmed into these image–dependent technologies that it often seems as if images are ‘thinking’ (see Burnett, How images think, 2004). In the VCR case it is as if the virtual replica of the hearings ‘channelled’ and shaped the thinking of the judges.
11. Thorburn and Jenkins, 2003, p. 4.
12. Dewey, 2002, p. 191. My emphasis in italics.
13. Strangeness may arise in two major ways: because ordinary objects exhibit surprising features and behaviors when they move to the new medium; and, because the new medium creates entire new objects of which there is no practical experience available yet. In the language of Heidegger’s hermeneutic phenomenology, objects and tools become present–at–hand or, as Dreyfus prefers to translate, convert to a mode of ‘occurrentness’ (Heidegger, 1962; Dreyfus, 1991). In this connection, in spite of the different philosophical interests and languages, there are quite a few substantial similarities between Dewey’s discussion of habit and Heidegger’s discussion of ‘being–in–the–world’ (Dewey, 2002). Both point to the practical and existential modalities through which we have experience of the world.
14. In this connection Bamberger and diSessa (2003) remark that the enigmatic nature of all representations originates from the difficulty and ambiguity of converting the experience of a phenomenon into a stable description that ‘holds still’ so that it can be handled.
15. In other words the new medium exposes what may be called the ‘representational fallacy’ — the effect by which through practice symbol–based, medium–specific entities become the objects that are thought to exist, that is, what we take to be knowledge (therefore unquestionably assuming that the representations and the medium in which they are embedded completely describe the objects, that which exists, or stands out).
Jeanne Bamberger and Andrea diSessa, 2003. “Music as embodied mathematics: A study of a mutually informing affinity,” International Journal of Computers for Mathematical Learning, volume 8, number 2, pp. 123–160.http://dx.doi.org/10.1023/B:IJCO.0000003872.84260.96
Marc Berg, 2000. “Lesson from a dinosaur: Mediating IS research through an analysis of the medical record,” In: Andy Sloane and Felix van Rijn (editors). Proceedings of the IFIP TC9 WG9.3 International Conference on Home Oriented Informatics and Telematics, “IF at Home: Virtual Influences on Everyday Life” Information, Technology and Society. Boston: Kluwer, pp. 487–506.
Jay David Bolter, 2001. Writing space: Computers, hypertext, and the remediation of print. Second edition. Mahwah, N.J.: Lawrence Erlbaum Associates.
Jay David Bolter and Richard Grusin, 1999. Remediation: Understanding new media. Cambridge, Mass.: MIT Press.
John Seely Brown and Paul Duguid, 1996. “The social life of documents,” First Monday,volume 1, number 1 (May), at http://firstmonday.org/htbin/cgiwrap/bin/ojs/index.php/fm/article/view/466/387, accessed 21 May 2010.
Ron Burnett, 2004. How images think. Cambridge, MA: MIT Press.
Daniel Chandler, 1996. “Engagement with media: Shaping and being shaped,” Computer–Mediated Communication Magazine (February), at http://www.aber.ac.uk/media/Documents/short/determ.html, accessed 21 May 2010.
H.M. Collins and T.J. Pinch, 1982. Frames of meaning: The social construction of extraordinary science. London: Routledge and K. Paul.
Scott D.N. Cook and John Seely Brown, 1999. “Bridging epistemologies: The generative dance between organizational knowledge and organizational knowing,” Organization Science,volume 10, number 4, pp. 38–-400.http://dx.doi.org/10.1287/orsc.10.4.381
Lorraine Daston (editor), 2004. Things that talk: Object lessons from art and science. New York: Zone Books.
John Dewey, 2002. Human nature and conduct. Mineola, N.Y.: Dover.
Paul Dourish, 1993. “Culture and control in media space,” Proceedings of the Third Conference on European Conference on Computer–Supported Cooperative Work (Milan), pp. 125–137.
Hubert Dreyfus, 1991. Being–in–the–world: A commentary on Heidegger’s Being and time. Cambridge, Mass.: MIT Press.
Clifford Geertz, 1983. Local knowledge: Further essays in interpretive anthropology. New York: Basic Books.
N. Katherine Hayles, 2002. Writing machines. Cambridge, Mass.: MIT Press.
Christian Heath and Paul Luff, 1992. “Collaboration and control: Crisis management and multimedia technology in London underground line control rooms,” Journal of Computer Supported Cooperative Work, volume 1, number 1, pp. 24–48.
Jannis Kallinikos, 2009. “On the computational rendition of reality: Artefacts and human agency,” Organization, volume 16, number 2, pp. 183–202.http://dx.doi.org/10.1177/1350508408100474
Jannis Kallinikos, 2006. “Information out of information: On the self–referential dynamics of information growth,” Information Technology and People, volume 19, number 1, pp. 98–115.http://dx.doi.org/10.1108/09593840610649989
Jannis Kallinikos, 2003. “Mediated action and representation: On the vicissitudes of human signification,” Homo oeconomicus, volume 19, number 4, pp. 607–622.
Sean Dorrance Kelly, 2005. “Seeing things in Merleau–Ponty,” In: Taylor Carman and Mark B.N. Hansen (editors). Cambridge companion to Merleau–Ponty. Cambridge: Cambridge University Press: pp. 74–110.
Friedrich A. Kittler, 1999. Gramophone, film, typewriter. Translated, with an introduction, by Geoffrey Winthrop–Young and Michael Wutz. Stanford, Calif.: Stanford University Press.
Friedrich A. Kittler, 1997. Literature, media, information systems: Essays. Edited and introduced by John Johnston. Amsterdam: GB Arts International.
Giovan Francesco Lanzara, 2009. “Reshaping practice across media: Material mediation, medium specificity and practical knowledge in judicial work,” Organization Studies, volume 30, number 12, pp. 1,369–1,390.
Giovan Francesco Lanzara, 1991. “Shifting stories: Learning from a reflective experiment in a design process,” In: Donald A. Schön (editor). The reflective turn: Case studies in and on educational practice. New York: Teachers College Press, pp. 285–320.
Giovan Francesco Lanzara and Gerardo Patriotta, 2001. “Technology and the courtroom: An inquiry into knowledge making in organizations,” Journal of Management Studies, volume 38, number 7, pp. 943–971.http://dx.doi.org/10.1111/1467-6486.00267
Lev Manovich, 2001. The language of new media. Cambridge, Mass.: MIT Press.
Steven Maras and David Sutton, 2000. “Medium specificity re–visited,” Convergence, volume 6, number 2, pp. 98–113.
Marshall McLuhan, 1964. Understanding media: The extensions of man. London: McGraw–Hill.
Reijo Miettinen, 2009. “Artifact mediation in Dewey and in cultural–historical activity theory,” Mind, Culture, and Activity, volume 8, number 4, pp. 297–308.http://dx.doi.org/10.1207/S15327884MCA0804_03
Reijo Miettinen, Dalvir Samra–Fredericks, and Dvora Yanow, 2009. “Re–turn to practice: An introductory essay,” Organization Studies, volume 30, number 12, pp. 1,309–1,327.
Wanda J. rlikowski and JoAnne Yates, 1994. “Genre repertoire: The structuring of communicative practices in organizations,” Administrative Science Quarterly, volume 39, number 4, pp. 541–574.http://dx.doi.org/10.2307/2393771
Seymour Papert, 1980, Mindstorms: Children, computers, and powerful ideas. New York: Basic Books.
Lydia Plowman, 1994. “The ‘primitive mode of representation’ and the evolution of interactive multimedia,” Journal of Educational Multimedia and Hypermedia, volume 3, numbers 3–4, pp. 275–293.
Michael Polanyi, 1967. The tacit dimension. New York: Anchor Books.
Donald A. Schön, 1983. The reflective practitioner: how professionals think in action. New York: Basic Books.
Robert J. Sternberg and Joseph A. Horwath (editors), 1999. Tacit knowledge in professional practice: Researcher and practitioner perspectives. Mahwah, N.J.: Lawrence Erlbaum.
Marita Sturken and Lisa Cartwright, 2001. Practices of looking: An introduction to visual culture. New York: Oxford University Press.
David Thorburn and Henry Jenkins, 2003. “Introduction: Towards an aesthetics of transition,” In: David Thorburn and Henry Jenkins (editors). Rethinking media change: The aesthetics of transition. Cambridge, Mass.: MIT Press, pp. 1–16.
Emmanuelle Vaast and Geoff Walsham, 2005. “Representations and actions: The transformation of work practices with IT use,” Information and Organization, volume 15, number 1, pp. 65–89.http://dx.doi.org/10.1016/j.infoandorg.2004.10.001
Lev Vygotsky, 1986. Thought and language. Cambridge, Mass.: MIT Press
Lev Vygotsky, 1978. Mind in society: The development of higher psychological processes. Cambridge, Mass.: Harvard University Press.
Paper received 4 May 2010; revised 11 May 2010; accepted 14 May 2010.
Copyright © 2010, First Monday.
Copyright © 2010, Giovan Francesco Lanzara.
Remediation of practices: How new media change the ways we see and do things in practical domains
by Giovan Francesco Lanzara.
First Monday, Volume 15, Number 6 - 7 June 2010
Instead of remediating a print text into a visual or audio text, you may choose to use a different genre within the print medium. For example, if your original text is a poem, you might want to remediate that poem into song lyrics, a children's book, a letter, or another print genre. Before you construct your text-to-text remediation, consider the following:
Before you can create any type of effective or meaningful remediation, you should develop a good understanding of your original text. Your remediation, after all, is based on your decoding of the messages and meanings—the textual content—of your original text. You would be hard-pressed to effectively argue, for example, that Martin Luther King Jr.'s "I Have a Dream" speech is a strongly worded argument against racial equality because even a basic understanding of King's speech illustrates that he was making the opposite point. In other words, you want to be sure that you have accurately captured the central idea(s) in the original text. Moreover, in the creation of your text-based remediation, you should suggest ideas that are similar to the ideas in the original text—if, to continue the MLK Jr., example, you choose to remediate the text of that particular speech into song lyrics, then you should capture the meaning of that speech (equality and a brighter future for the next generation, perhaps) in the lyrics of your song.
Ask yourself the following question: What is the main purpose of the text'? If the original text is a poem that seems to advocate an anti-war message, you can surmise that the poem's purpose is to capture a reader's attention through attractive writing and to persuade the reader that war is to be avoided. There is a distinct connection between a text's purpose and the medium in which it is presented: an opinion column in a newspaper is traditionally a type of text where serious issues are addressed with a central purpose of convincing others to agree with a particular point of view. In this way, medium and purpose are inextricably linked. As you analyze the text that you want to remediate, try to express the purpose of this text: Does it aim to entertain, to enrage, to convince, or cause reflection? And, more to the point, in what ways is that purpose reinforced or modified by the medium that the text uses?
Everything is a Remix Part 1 from Kirby Ferguson on Vimeo.
Likewise, when you are creating your text-based remediation, think about how you want to manipulate the purpose of the text. A blog post in an online forum that urges people to stop the Iraq war could be remediated in a number of ways, but be mindful that the main purpose of that text is to persuade or convince—a purpose that demands a different rhetorical approach than a text which wishes only to entertain. What this example illustrates, hopefully, is that purpose is inseparable from medium, audience, and rhetorical stance, and an awareness of purpose should be something which governs a writer's remediation process.
An equally important consideration in the construction of a text-to-text remediation is identifying the possible audiences of both the original text and the remediation. Subtle changes in the delivery method of text—poem, newspaper article, Twitter post—will appeal to very different audiences. This point is very important and plays a major role in the creation of your remediation. An understanding of the characteristics of your audience will help you to identify the tone, word choice, and voice you'll adopt as you remediate the text.
An example might make this point clearer: let's say you remediate a newspaper article about gun control into a series of tweets. You, as a sharp and engaged student, realize these two text-based media have decidedly different target audiences: a newspaper article has an older, perhaps less technologically savvy audience, while the Twitter page is aimed at a younger, more technologically comfortable audience. This difference in target audience must play a large role in your rhetorical stance—word choice, phrasing and voice— so you choose to adopt in your remediation.
Newspaper article phrase:
- “The decline of the quality of some high schools has people concerned”
- “High School = brain death + pep rallies?”
You might notice how different the rhetorical approaches are in these two text phrases (different word choice, different tone, etc.); however, the fundamental idea or message has remained largely unchanged. Moreover, the likely audience for a Twitter post is altogether different than that of a newspaper article. The rhetorical stance used in the Twitter post, of course, reflects this difference.
We instinctively know how to speak or write in vastly different ways when addressing various audiences; you would, for example, hardly address your first-year composition instructor in the same way you would your roommate. Something similar is at work when you compose your remediation: you need to identify the expected audience for your text-based remediation and allow your audience's expectations to dictate the way you go about creating the new version of the text.
A text-to-text remediation requires a more subtle and analytical approach than a visual remediation or a multimodal remediation because the former involves only small changes in medium. It's important to carefully consider the original text and your subsequent remediation through a thorough analysis of meaning, target audience, purpose, and the way in which rhetorical stance is affected by all three of these concerns. To be an effective writer, you should know that huge changes in medium need not be the only way in which a text's meaning can be significantly altered; instead, a text-based remediation draws its creative significance from tinkering with textual content, targeting new audiences, and recalibrating the original text's purpose.