Tag: methods

Posted on

Are Online Communities Places or Artifacts?

I’m sympathetic to two ways of thinking about online communities that are somewhat inconsistent with each other. The web is a stack of communication technologies (both software and hardware) and should be studied in the same way that one would study the pony express, telegraphy, or the book. Yet, the web has communities, things that through ongoing social interaction where people spatialize the communication technology to “lurk” “hang out” and talk about the other kinds of people that do things different over there.  Online communities end up feeling like places and when we interact with people who are similar in some ways and different in others in those places we end up with cultures.

The Myth of Cyberspace and Possibility of Being There

I full well realize that the web isn’t a space. I’m with PJ Ray on the entire Myth of Cyberspace.  It doesn’t have dimensions, it is a stack of technologies (hardware and software). More specifically it is a constellation of technologies assembled in different arrangements by different individuals. However that stack/constellation  clearly creates cultures. Now sure books create cultures, telephones create cultures, and the postal service creates cultures. With that said, those republics of letters, and literary cultures aren’t really the same kinds of culture that one studies in an ethnography. I mean, imagine pen-pal-nography, telegram-nography—they just sound wrong. You can talk about a republic of letters all you like, but the moment you start saying you are doing an ethnography of letters someone is going to tell you you’re doing it wrong. When you study letters you are studying documents. We study documents as a species of artifact. Yes,  we learn about culture through that study (that would be the entire idea of material culture), but we don’t think of reading letters as “participant observation.”

With all this said, I still think the idea of “netnography” totally makes sense in a way that all those other –nographies doesn’t. Something about the medium of the web (I’d hazard its’ immediacy, two-way-nature, the placey-ness of URLs as locations) ends up giving us the things that we need to think about it as a place and gives us the experiences that we need to really make cultures happen. That is, we are thrown into a thing that works like proximity to others in which we interact with them and develop some shared ways of being in the world while retaining a whole host of dissonant and contradictory feelings about things.

Putting the Field in Computer Mediated Field Work

If you are unfamiliar with the idea of netnography I would suggest Kozinets book, Netnography: Doing Ethnographic Research Online. In contrast to the idea of “virtual ethnography” Kozinets is part of a group of researchers who gets behind the idea of “netnography.” (Rightly these folks acknowledge that there is nothing “virtual” about the web, it’s a real thing). The decision to shift to use netnography instead of ethnography comes from a sense that studying online communities is so substantively different from studying them in physical space that it needs a whole different term. That is, you can study how existing communities use the web alongside other modes of interaction, but there are also communities that exist solely as a result of particular web forums, listservs, and such.

In the last few weeks I’ve read and re-read Netnography, switching between modes of enthusiastic underlining (YES! That is it!). For example, when Kozinets talks about “alteration” recognizing that in online communities “the nature of the interaction is altered—both constrained and liberated—by the specific nature and rules of the technological medium in which it is carried.” (68) However, there are other moments in which I scrawl disapproving marginalia. For example, when I see terms like “online-fieldsite” (NO, the web is not a place and we shouldn’t pretend it is!). I think I can get behind “computer mediated fieldwork,” which he uses in other places, but I’m not sure I can go to “fieldsite.”

Can we talk of “Participant Observation” when we aren’t observing people?

I’ve gone back and forth in my head about Kozinets idea that we do “participant observation” when we study interactions in an online community. How can we talk of observing participants when we are actually observing artifacts? He suggests that our actions in online communities, our clicks, our keystrokes, are effectively utterances. Which is true, but at the same time when we study those utterances it isn’t like when we experience someone talking to us, documents are being created and we are reading them. It is effectively the same as reading a letter. Still, I think those specific features of the web mediums end up making this a situation where we can get away with the “participant observation” metaphor. Yes, if a netnographer jumps into an online community and starts to engage in the ebb and flow of exchange they are doing something that may have more in common with direct participation than with the hermeneutic interpretation of documents.

Theorizing and Interpreting Kinds of Online Community Data

Kozinats discusses three types of data. Archival data (data copied from “pre-existing computer-mediated communications of online community members), Elicited data, (data co-created with “culture members through personal and communal interaction”) and Fieldnote data (the researchers  own notes, observations and self reflections). He suggests that his categories are  similar to Wolcott’s notions of qualitative researchers “watching, asking and examining” and Miles and Huberman’s focus on studying “documents, interviews and observations” as kinds of data to interpret.  These are potentially useful comparisons, and as we need to come up with ways to fit new things into old boxes to make sense of them I can get behind the impulses here.

What’s at issue here is how much the experience of participating in an online community is like participating in a communities that occupy physical space. I think this is particularly tricky in that some of the features that make the web a rather unique medium are the things that give online communities their place-like qualities. To attend to the mediality of the web is to recognize it has this set of place-like or place-affording qualities.

“Archival data” Transcript, Recording, or Encoding

Kozinats struggles a bit to explain “archival data,” not that it is data that is being collected and organized by an archive, but in the much more nebulous sense of archival that has come to mean old-stuff-that-is-still-around-for-some-reason.  At one point, he suggests that the wide availability of this archival data in previous discussion on the boards or old email threads from listservs would be equivalent of “every public conversation being recorded and made available as transcripts.” However, importantly, a listserv archive, and old posts to discussion boards are not “recordings” of what transpired, they are what transpired. The creation of the “archive” is to some extent embedded in the act of communicating through these mediums. With that said, if you aren’t experiencing these exchanges as they happen then there are going to be issues that require you to reconstruct context and make sure that what you are looking at is authentically what was created at the time you want to make inferences about. That is, people edit their posts on discussion boards, users delete their accounts and the contextual information about who they were is often erased, site administrators prune away or remove posts over time. Generally, what we colloquially call an archive with these kinds of online communities is really a pile of things that have some connection to the past but haven’t really been worked over or documented. In any event, it is critical to not take for granted that you are looking at accurate recordings of the past, but to think about the provenance and particular constellations of technologies and users that made it possible for you to look at recordings of previous interactions between members of an online community.

So what can we do with these records of discourse? Kozinats suggests that  “Archival cultural data provide what amounts to a cultural baseline. Saved communal interactions provide the netnographer with a convenient bank of observational data that may stretch back for years.” (104) I’m not sure that this works. I don’t think we can talk about this archival data as “observational data.” It is not something you observed it is a set of documentary evidence that you need to establish the provenance and context of and can then engage in interpreting in the way a historian interprets any textual records. When it isn’t currently happening you aren’t observing it. These utterances become documents as they slide out of the present and into the past.

So are Online Communities Places or Objects

I feel like the answer here has to be something like, they are objects (or specifically assemblages of hardware and software technologies and protocols) that produce place-like experiences. So, it makes sense to try and figure out what it is like to be “a redditor” or to study how redditors interact with eachother and the kinds of communities that emerge there. With that said, reddit isn’t a fieldsite. Reddit is software, a database, and a set of bits on a series of servers accessible over HTTP.

All of that stuff, those objects create and log communication in such a way that they take on place-like qualities. People lurk in some sub reddits, they build relationships with the folks they come into contact with, they develop some shared and conflicting ideas about the world. In short, people create cultures through the affordances of the technologies. That cultural component, the way people use these things, gets rolled back into changing the structure and nature of the technologies that afford the place like qualities.

A Note on Determinisms and Co-Construction

Importantly, this does not mean that they “co-construct” each other. Kozinets nods to this in the beginning of the book. The idea that the forces of technological determinism and social construction of technology have come together in a kumbaya moment where technology and culture each construct each other feels too wishy-washy. Objects and artifacts afford and resist, people interact and interpret (often drawing on their own cultural tool kits or their internal representations of generalized others) and the social or the cultural emerges through this network of actors and actants. That’s at least my best stab at this for now. So yes, it’s not an either or, but I think it’s too much of a gloss to say its co-construction

Open questions?

I’d love to hear how other folks parse out these distinctions. What kind of thing is an online community and where are the limits of talking about them as places, as cultures, as technologies and as documents? Do you agree with how I am parsing this out? Or do you think I’m way off base here?

Posted on

The Value of Design Narratives: The Case of Environmental Detectives

In Please Write it Down: Design and Research in the Digital Humanities I suggested that there are some valuable ways of thinking about the connections between building/designing and creating knowledge and scholarship.  In particular, I suggested that those interested in learning through building in the digital humanities might find some value in work in educational research over the last decade which has tried to define what exactly what a design based research methodology might look like.

This is the first post, in what I imagine might be an ongoing line of thought here, to try to put ideas from design based research in conversation with the digital humanities. As a point of entry, I am going to walk through one emerging genre of writing in design based research, the design narrative. Before getting there, however, I would briefly pause to note that the journal this piece appeared in, Educational Technology Research and Development, is itself an interesting note to the digital humanities. I for one, would love to see a journal in the digital humanities similarly situated as a place for sharing and disseminating R&D knowledge.

The Case of Environmental Detectives

In Environmental Detectives: The Development of an Augmented Reality Platform for Environmental Simulations Eric Klopfer and Kurt Squire offer a summative and reflective report on their work developing the augmented reality game Environmental Detectives. The paper makes some valuable suggestions for how we might better design augmented reality games, but I think its primary strength is as an example of a particularly novel and useful genre of design based research report. 

Brenda Bannan-Ritland’s article, The role of design in research: The integrative learning design framework offers a robust framework for thinking through how the design process and the research process can fit together. See her diagram below  (don’t get lost in the details). The intellectual work that diagram and her approach offers os to illustrate what happens if you mush together the steps in an array of design processes and research approaches. The diagram illustrates how the features of product development, research design, and user centered design can leaf together. 

If you look a the top part of the diagram carefully you will notice that practically every step in this process has an arrow that points over to the publish results box. This is a key concept here, the idea behind design based research is not that the design process is itself a research method, but that throughout the design process there are a series of publishable results and lessons learned that emerge which warrant being refined, shared and communicated. Squire and Klopfer’s article is a great example of the kind of piece one would want to write as a summative result of an extended design research process.

Design Narrative as a Genre of Design Based Research Article

Design based research can generate publishable results in any particular research tradition. You can find interviews, ethnographic approaches, micro ethnographic approaches,  case studies, randomized clinical trials, and methods from usability studies like eye tracking used at different points in the design and development process. In short, there are any number of ways to use existing research methods approaches to reflect on and report out results of research in the process of informing design. Part of what is particularly interesting about Klopfer and Squire’s paper is that it represents a somewhat novel mode of research writing, the design narrative.

Drawing from Hoadley’s 2002 piece, Creating context: Design-based research in creating and understanding CSCL, Klopfer and Squire offer a reflective narrative account of their work designing, developing and researching the Environmental Detectives game. Unlike other papers they published, which might report parts of this research in terms of a case study, or the pre-post test scores or the results of a particular evaluative test of the game’s outcomes, this summitive piece serves to reflect on the design process and offer an account of the context and lessons learned in the course of the design process. It is worth reporting on actual structure of the piece.

Review of literature that informed the design: After explaining background on the idea of design narrative Klopfer and Squire offer an account of both the extent literature on augmented reality games and a review of the existing games projects that they looked to which informed their design. This serves to provide the conceptual context that they began from, it sets the reader up to understand exactly where the project started from while also providing information on what theory and knowledge at the time of the projects start looked like.

Retrospective and Reflective Design Narrative: The bulk of the paper then reports out on each phase of their design process. In their particular case they describe six phases of their research, brainstorming, designing the first instantiation, developing a first generation prototype, classroom field trials, classroom implementations, expanding to new contexts, and a sixth phase in which they added customized dynamic events to the game. It is not necessary to go into the details of each section for this review. What matters is to stress that each section begins by explaining how they went about their work in the given phase and reports a bit on what they learned in that phase. What is essential in this approach is that each section explains what worked and didn’t work in any given phase and how exactly Klopfer decided to remedy their approach and design to respond to problems.

As is generally the case with qualitative research, the moments when things don’t go according to plan and exactly how we make sense and work through those moments are generally the most valuable parts of the process. The value in this kind of retrospective account is two-fold. It provides a context for understanding why the game they made does what it does, but more importantly, the design narrative’s primary value is as a guide to other designers on what parts of the design process were particularly valuable. This kind of narrative helps us to refine our ideas not only about this particular design situation, but more broadly about how we can refine our own design practices.

Conclusions and Implications from Reflection: After reporting the design narrative the paper presents a set of technological and pedagogical implications. In much the way that the discussion section and conclusion sections of research reports function, this section attempts to suss out and distill the lessons learned from the work. In their case, they present a range of specific implications for the design of augmented reality games that emerged from their design approach.

The Value of Design Narratives

If you read through their references, you can see that they have published about this work on a few previous occasions. It is not that they are double dipping on publications, instead those other publications report results from subsets of this project, some of the earlier findings, or any of the points in the design process that resulted in interesting findings. This paper is really a summitive report, retracing the design narrative of the entire project.
I see the value of this particular design narrative approach as having two primary values, two values that I think are particularly useful to the still emerging world of the digital humanities. Composing these narratives serves an internal value to designers as part of reflective practice. Sharing these narratives makes the kinds essential tacit knowledge that comes about as part of doing design accessible to others.

Reflective Practice is Best Practice: If you can hold yourself to some sound practices for documenting the stages in your design process (the ideas that you had, how you went about implementing and revising them, and the results), you are in a good position to use that documentation to reflect on your practice. In this sense, the design narrative, the retrospective account of what you did, why you did it, what you learned  is an essential piece of doing reflective design practice. When you go back and think through your own process you are not simply reporting on what you learned you are actually making sense out of your trajectory and coming to understand what it is that you actually learned. Like much of qualitative and hermeneutic research, the process of writing is not a process of transmission of knowledge but of the discovery of knowledge. Writing a design narrative is the process by which we come to know and learn from our work.

Making Tacit Practical Design Knowledge Explicit and Available: It is essential that the knowledge developed in the design process is documented and shared. While the individual studies that come out of a design research process provide evidence of the value, or of particular lessons learned in part of a design project, they leave a considerable amount of the bigger picture knowledge off the table. Quite frankly, much of the most essential parts of design are not about explaining that something works, if someone wants to get into design they need access to the deeply pragmatic, heuristic driven, knowledge that develops on over time in the process of design. The design narrative is an essential medium for capturing and disseminating this kind of tacit knowledge.

In short, I would suggest that this particular piece of scholarship serves as a great example of the value of reporting design narratives and an exemplar for others to use as a model for composing their own design narratives.