The Value of Design Narratives: The Case of Environmental Detectives

In Please Write it Down: Design and Research in the Digital Humanities I suggested that there are some valuable ways of thinking about the connections between building/designing and creating knowledge and scholarship.  In particular, I suggested that those interested in learning through building in the digital humanities might find some value in work in educational research over the last decade which has tried to define what exactly what a design based research methodology might look like.

This is the first post, in what I imagine might be an ongoing line of thought here, to try to put ideas from design based research in conversation with the digital humanities. As a point of entry, I am going to walk through one emerging genre of writing in design based research, the design narrative. Before getting there, however, I would briefly pause to note that the journal this piece appeared in, Educational Technology Research and Development, is itself an interesting note to the digital humanities. I for one, would love to see a journal in the digital humanities similarly situated as a place for sharing and disseminating R&D knowledge.

The Case of Environmental Detectives

In Environmental Detectives: The Development of an Augmented Reality Platform for Environmental Simulations Eric Klopfer and Kurt Squire offer a summative and reflective report on their work developing the augmented reality game Environmental Detectives. The paper makes some valuable suggestions for how we might better design augmented reality games, but I think its primary strength is as an example of a particularly novel and useful genre of design based research report. 

Brenda Bannan-Ritland’s article, The role of design in research: The integrative learning design framework offers a robust framework for thinking through how the design process and the research process can fit together. See her diagram below  (don’t get lost in the details). The intellectual work that diagram and her approach offers os to illustrate what happens if you mush together the steps in an array of design processes and research approaches. The diagram illustrates how the features of product development, research design, and user centered design can leaf together. 

If you look a the top part of the diagram carefully you will notice that practically every step in this process has an arrow that points over to the publish results box. This is a key concept here, the idea behind design based research is not that the design process is itself a research method, but that throughout the design process there are a series of publishable results and lessons learned that emerge which warrant being refined, shared and communicated. Squire and Klopfer’s article is a great example of the kind of piece one would want to write as a summative result of an extended design research process.

Design Narrative as a Genre of Design Based Research Article

Design based research can generate publishable results in any particular research tradition. You can find interviews, ethnographic approaches, micro ethnographic approaches,  case studies, randomized clinical trials, and methods from usability studies like eye tracking used at different points in the design and development process. In short, there are any number of ways to use existing research methods approaches to reflect on and report out results of research in the process of informing design. Part of what is particularly interesting about Klopfer and Squire’s paper is that it represents a somewhat novel mode of research writing, the design narrative.

Drawing from Hoadley’s 2002 piece, Creating context: Design-based research in creating and understanding CSCL, Klopfer and Squire offer a reflective narrative account of their work designing, developing and researching the Environmental Detectives game. Unlike other papers they published, which might report parts of this research in terms of a case study, or the pre-post test scores or the results of a particular evaluative test of the game’s outcomes, this summitive piece serves to reflect on the design process and offer an account of the context and lessons learned in the course of the design process. It is worth reporting on actual structure of the piece.

Review of literature that informed the design: After explaining background on the idea of design narrative Klopfer and Squire offer an account of both the extent literature on augmented reality games and a review of the existing games projects that they looked to which informed their design. This serves to provide the conceptual context that they began from, it sets the reader up to understand exactly where the project started from while also providing information on what theory and knowledge at the time of the projects start looked like.

Retrospective and Reflective Design NarrativeThe bulk of the paper then reports out on each phase of their design process. In their particular case they describe six phases of their research, brainstorming, designing the first instantiation, developing a first generation prototype, classroom field trials, classroom implementations, expanding to new contexts, and a sixth phase in which they added customized dynamic events to the game. It is not necessary to go into the details of each section for this review. What matters is to stress that each section begins by explaining how they went about their work in the given phase and reports a bit on what they learned in that phase. What is essential in this approach is that each section explains what worked and didn’t work in any given phase and how exactly Klopfer decided to remedy their approach and design to respond to problems.

As is generally the case with qualitative research, the moments when things don’t go according to plan and exactly how we make sense and work through those moments are generally the most valuable parts of the process. The value in this kind of retrospective account is two-fold. It provides a context for understanding why the game they made does what it does, but more importantly, the design narrative’s primary value is as a guide to other designers on what parts of the design process were particularly valuable. This kind of narrative helps us to refine our ideas not only about this particular design situation, but more broadly about how we can refine our own design practices.

Conclusions and Implications from Reflection: After reporting the design narrative the paper presents a set of technological and pedagogical implications. In much the way that the discussion section and conclusion sections of research reports function, this section attempts to suss out and distill the lessons learned from the work. In their case, they present a range of specific implications for the design of augmented reality games that emerged from their design approach.

The Value of Design Narratives

If you read through their references, you can see that they have published about this work on a few previous occasions. It is not that they are double dipping on publications, instead those other publications report results from subsets of this project, some of the earlier findings, or any of the points in the design process that resulted in interesting findings. This paper is really a summitive report, retracing the design narrative of the entire project.
I see the value of this particular design narrative approach as having two primary values, two values that I think are particularly useful to the still emerging world of the digital humanities. Composing these narratives serves an internal value to designers as part of reflective practice. Sharing these narratives makes the kinds essential tacit knowledge that comes about as part of doing design accessible to others.

Reflective Practice is Best Practice: If you can hold yourself to some sound practices for documenting the stages in your design process (the ideas that you had, how you went about implementing and revising them, and the results), you are in a good position to use that documentation to reflect on your practice. In this sense, the design narrative, the retrospective account of what you did, why you did it, what you learned  is an essential piece of doing reflective design practice. When you go back and think through your own process you are not simply reporting on what you learned you are actually making sense out of your trajectory and coming to understand what it is that you actually learned. Like much of qualitative and hermeneutic research, the process of writing is not a process of transmission of knowledge but of the discovery of knowledge. Writing a design narrative is the process by which we come to know and learn from our work.

Making Tacit Practical Design Knowledge Explicit and Available: It is essential that the knowledge developed in the design process is documented and shared. While the individual studies that come out of a design research process provide evidence of the value, or of particular lessons learned in part of a design project, they leave a considerable amount of the bigger picture knowledge off the table. Quite frankly, much of the most essential parts of design are not about explaining that something works, if someone wants to get into design they need access to the deeply pragmatic, heuristic driven, knowledge that develops on over time in the process of design. The design narrative is an essential medium for capturing and disseminating this kind of tacit knowledge.

In short, I would suggest that this particular piece of scholarship serves as a great example of the value of reporting design narratives and an exemplar for others to use as a model for composing their own design narratives.

Reading the Sporum: What Players do with Spore

It appears the stars have aligned and several papers I have had in the works for a while are hitting the streets at the same time. I’m excited to announce that an article I wrote for Cultural Studies of Science Education is now up in “Online First.” I thought I would share the abstract, and one section from the paper where I talk through some of the ways that people role play at natural history in the forums.

Owens, T. J. (2012). Teaching intelligent design or sparking interest in science?
What players do with and take away from Will Wright’s Spore. Cultural Studies of Science Education  DOI: 10.1007/s11422-012-9383-5

Abstract: Teaching intelligent design or sparking interest in science?

The 2008 commercial video game Spore allowed more than a million players to design their own life forms. Starting from single-celled organisms players played through a caricature of natural history. Press coverage of the game’s release offer two frames for thinking about the implications of the game. Some scientists and educators saw the game as a troubling teacher of intelligent design, while others suggested it might excite public interest in science. This paper explores the extent to which these two ways of thinking about the game are consistent with what players have done with the game in its online community. This analysis suggests that, at least for the players participating in this community, the game has not seduced them into believing in intelligent design. Instead the activities of these players suggest that the game has played a catalytic role in engaging the public with science. These findings indicate that designers of educational games may wish to consider more deeply tensions between prioritizing accuracy of content in educational games over player engagement.

The Evolution of the Javelin Hawk

I had a ton of fun with this paper. The Sporum, the Spore web forums, is a crazy place and a good time. I think it also turns out to be a great place to test ideas about what people who play sand box games like Spore end up doing as a result of their experiences playing the game. In any event, writing about the “Javelin Hawk” will likely be my only chance to discuss something that has a “prehensile throat” which it uses to “spear live prey and drink its innards using gastric juices vomited up through the throat”. Enjoy!

Stylistically written to evoke what might be described as, ‘‘textbook language’’ a player describes how the ‘‘Javelin Hawk evolved from the Archeopteryx, an early bird.’’ The player explains, Archeopteryx developed a ‘‘strange mutation in which part of the esophagus was extended into the mouth, resembling a hose.’’ It is important to note that the game itself does not employ the idea of mutation. In Spore, players spend ‘‘DNA points’’ to add features to their creatures. This player brought in the idea of mutation to serve as a layer of explanation for how their creature came about. The player goes on to explain that this mutation ‘‘was usually fatal, until the opening evolved to be prehensile.’’ This is, yet again, a significant addition to the way the game works. Not only is the player using the idea of mutation, she is also presenting mutation as something that, for most of the creatures who exhibited the mutation, is fatal. Only coupled with an additional mutation, the player explains, did these creatures’ esophagi became prehensile resulting in a viable new species. The player explains and names this creature as an intermediate form; ‘‘This creature with a prehensile throat was known as the Perlingua.’’ From there, the player reports, ‘‘Eventually, with the extinction of the succulent plants it fed on in the area, it evolved a larger longue [sic] that was very sharp to spear live prey and drink it’s [sic] innards using gastric juices vomited up through the throat.’’ In this explanation, the player identifies that the loss of the creatures’ food source, the plants, led to it ‘‘evolving’’ a larger tongue with which it could spear live prey. The description here sounds a bit Lamarkian: the extinction of the plants that the creatures ate could have led to their extinction but could not prompt them to ‘‘evolve.’’ Instead, a loss of a creatures’ food source could act as a factor in natural selection.

For those without access to the official copy you can see my personal unofficial archival HTML copy here on my website.

Deforming reality with Word Lens

If you haven’t checked it out already Wordlens is an amazingly cool iPhone app that will automatically translate text on the fly, as you see it.

I’ve had it on my phone for about a month now, but I find that the things it messes up are far more interesting than the things it gets right. Messes up is really the wrong term here. The best parts of wordlens happen when you point it at things you arn’t supposed to point it at or that arn’t in the language you are supposed to be translating.

When you hold it up and pan around your environment it is like the software is uncovering the hidden meanings in your environment. For example, I pointed it at some of the congressional buildings on my walk home and was told that “NEICAH” was apparently “IN”. 
It is a jarring experience to walk around and see these words keep poping up, as if they emerge out of the environment. After using it for a bit you get a handle for what kinds of things you can trick it into thinking are text.

You want to have some clear horizontal lines, but beyond that you want a visual space with some clear visual breaks in it. For example, a flower bed worked great. I couldn’t help thinking that it would be really neat if they would create some explicit vocabulary packs that were focused on this off purpose use. If instead of simply translating text the Wordlens developers gave us a few more fun ways to try and deform and uncover hidden meaning and jokes in reality.

Eventually, I went out and bought the Spanish to English pack. I wanted to see what kind of things it would see when it was working off a English vocabulary. That is when I realized that the Wordlens developers had already given us everything we need. Just flip it on to try and turn Spanish into English and refuse to show it any Spanish and you have your self something between a decoder ring and a reading machine that you can turn to deform any text or potential text for fun and profit. Ok, no profit, but lots of fun. Possibly insight. You can see some of the results of that in the gallery. I most enjoyed what happened when I turned it on some of my books. The following examples are Wordlens attempting to translate books with English language titles from Spanish into English.

Wordlens can be Snarky and Potentially Insightful

I thought some of these were rather funny. When exposed to the Spanish to English filter Debates in the Digital Humanities became “DEBATES IN THE OR DIGITAL ROYALTY.” Something that is particularly humorous given discussion of the digital humanities cool kids table. It felt a little bit like Mark’s “Hacking the Accident” moment. The machine is mangling the text and that deformed text provokes thought and consideration.

Observing the User Experience became Observing Was User experience. Which is in fact totally true. Observers are themselves users observing other users.

Wordlens seems to disagree with Latour’s Actor Network Theory, which it calls “THE Actor-N ERRORS THEORY.” Or I suppose this might actually be a totally different book, one written by Bruno Brassr called “Reassembling Read Social” in which we are introduced to the brand new Actor-NERRORS Theory.

I have saved the best for last. In what seemed particularly topical, Steve Ramsey’s Reading Machines becomes Reading Machetes. Even better, when we flip to the back of the book we learn that it is part of the Mythical Theory Reiterate Studies series. Based off his “CREEP” essay “Toward an Algorithmic Criticism.” From there I think I lose it a bit. Something about his “Thai Wrath.” With that said, I love that literary computing becomes Liberary computing which I assume is a mixture of liberation and library. Importantly, the back of Reading Machetes mentions the GNU operating system, liberary computing at its best.  It is also apparently “Trying” to “Shame” other scholars for their “LETHARGY” Criticism. Ha!

Here is a gallery of a few more images: