Designing Online Communities: Read My Accepted Dissertation Proposal

Wisdom of the Ancients: the web-comic-epigraph for my dissertation proposal, from XKCD

As of last monday, I have now successfully defended my dissertation proposal. In the context of my doctoral program, that means there is just one more hurdle to climb over to finish. I’m generally rather excited about the project, and would be thrilled to have more input and feedback on it (Designing Online Communities Proposal PDF). I would be happy for any and all comments on it in the comments of this post.

Designing Online Communities: How Designers, Developers, community Managers, And Software Structure Discourse And knowledge Production On The Web

Abstract: Discussion on the web is mediated through layers of software and protocols. As scholars increasingly turn to study communication, learning and knowledge production on the web, it is essential to look below the surface of interaction and consider how site administrators, programmers and designers create interfaces and enable functionality. The managers, administrators and designers of online communities can turn to more than 20 years of technical books for guidance on how to design and structure online communities toward particular objectives. Through analysis of this “how-to” literature, this dissertation intends to offer a point of entry into the discourse of design and configuration that plays an integral role in structuring how learning and knowledge are produced online. The project engages with and interprets “how-to” literature to help study software in a way that respects the tension that exists between the structural affordances of software with the dynamic and social nature of software as a component in social interaction.

What’s Next? 

At some point in the next year I will likely defend a completed dissertation. Places do dissertations differently, in my program the idea is that what I just defended is actually the first three chapters of a five chapter dissertation. So, at this point I need to follow through on what I said I would do in my methods section (to create chapter 4, results) and then write up how it connects with the conceptual context section (to create chapter 5, conclusions). So I should be able to grind this out in relatively short order.

At this point, I think this project should be interesting enough to warrant a book proposal. So I’ll likely start exploring putting together a book proposal for it in the next year as well. With that in mind, any suggestions for who might be interested in receiving a proposal on this topic are welcome.

The Value of Design Narratives: The Case of Environmental Detectives

In Please Write it Down: Design and Research in the Digital Humanities I suggested that there are some valuable ways of thinking about the connections between building/designing and creating knowledge and scholarship.  In particular, I suggested that those interested in learning through building in the digital humanities might find some value in work in educational research over the last decade which has tried to define what exactly what a design based research methodology might look like.

This is the first post, in what I imagine might be an ongoing line of thought here, to try to put ideas from design based research in conversation with the digital humanities. As a point of entry, I am going to walk through one emerging genre of writing in design based research, the design narrative. Before getting there, however, I would briefly pause to note that the journal this piece appeared in, Educational Technology Research and Development, is itself an interesting note to the digital humanities. I for one, would love to see a journal in the digital humanities similarly situated as a place for sharing and disseminating R&D knowledge.

The Case of Environmental Detectives

In Environmental Detectives: The Development of an Augmented Reality Platform for Environmental Simulations Eric Klopfer and Kurt Squire offer a summative and reflective report on their work developing the augmented reality game Environmental Detectives. The paper makes some valuable suggestions for how we might better design augmented reality games, but I think its primary strength is as an example of a particularly novel and useful genre of design based research report. 

Brenda Bannan-Ritland’s article, The role of design in research: The integrative learning design framework offers a robust framework for thinking through how the design process and the research process can fit together. See her diagram below  (don’t get lost in the details). The intellectual work that diagram and her approach offers os to illustrate what happens if you mush together the steps in an array of design processes and research approaches. The diagram illustrates how the features of product development, research design, and user centered design can leaf together. 

If you look a the top part of the diagram carefully you will notice that practically every step in this process has an arrow that points over to the publish results box. This is a key concept here, the idea behind design based research is not that the design process is itself a research method, but that throughout the design process there are a series of publishable results and lessons learned that emerge which warrant being refined, shared and communicated. Squire and Klopfer’s article is a great example of the kind of piece one would want to write as a summative result of an extended design research process.

Design Narrative as a Genre of Design Based Research Article

Design based research can generate publishable results in any particular research tradition. You can find interviews, ethnographic approaches, micro ethnographic approaches,  case studies, randomized clinical trials, and methods from usability studies like eye tracking used at different points in the design and development process. In short, there are any number of ways to use existing research methods approaches to reflect on and report out results of research in the process of informing design. Part of what is particularly interesting about Klopfer and Squire’s paper is that it represents a somewhat novel mode of research writing, the design narrative.

Drawing from Hoadley’s 2002 piece, Creating context: Design-based research in creating and understanding CSCL, Klopfer and Squire offer a reflective narrative account of their work designing, developing and researching the Environmental Detectives game. Unlike other papers they published, which might report parts of this research in terms of a case study, or the pre-post test scores or the results of a particular evaluative test of the game’s outcomes, this summitive piece serves to reflect on the design process and offer an account of the context and lessons learned in the course of the design process. It is worth reporting on actual structure of the piece.

Review of literature that informed the design: After explaining background on the idea of design narrative Klopfer and Squire offer an account of both the extent literature on augmented reality games and a review of the existing games projects that they looked to which informed their design. This serves to provide the conceptual context that they began from, it sets the reader up to understand exactly where the project started from while also providing information on what theory and knowledge at the time of the projects start looked like.

Retrospective and Reflective Design NarrativeThe bulk of the paper then reports out on each phase of their design process. In their particular case they describe six phases of their research, brainstorming, designing the first instantiation, developing a first generation prototype, classroom field trials, classroom implementations, expanding to new contexts, and a sixth phase in which they added customized dynamic events to the game. It is not necessary to go into the details of each section for this review. What matters is to stress that each section begins by explaining how they went about their work in the given phase and reports a bit on what they learned in that phase. What is essential in this approach is that each section explains what worked and didn’t work in any given phase and how exactly Klopfer decided to remedy their approach and design to respond to problems.

As is generally the case with qualitative research, the moments when things don’t go according to plan and exactly how we make sense and work through those moments are generally the most valuable parts of the process. The value in this kind of retrospective account is two-fold. It provides a context for understanding why the game they made does what it does, but more importantly, the design narrative’s primary value is as a guide to other designers on what parts of the design process were particularly valuable. This kind of narrative helps us to refine our ideas not only about this particular design situation, but more broadly about how we can refine our own design practices.

Conclusions and Implications from Reflection: After reporting the design narrative the paper presents a set of technological and pedagogical implications. In much the way that the discussion section and conclusion sections of research reports function, this section attempts to suss out and distill the lessons learned from the work. In their case, they present a range of specific implications for the design of augmented reality games that emerged from their design approach.

The Value of Design Narratives

If you read through their references, you can see that they have published about this work on a few previous occasions. It is not that they are double dipping on publications, instead those other publications report results from subsets of this project, some of the earlier findings, or any of the points in the design process that resulted in interesting findings. This paper is really a summitive report, retracing the design narrative of the entire project.
I see the value of this particular design narrative approach as having two primary values, two values that I think are particularly useful to the still emerging world of the digital humanities. Composing these narratives serves an internal value to designers as part of reflective practice. Sharing these narratives makes the kinds essential tacit knowledge that comes about as part of doing design accessible to others.

Reflective Practice is Best Practice: If you can hold yourself to some sound practices for documenting the stages in your design process (the ideas that you had, how you went about implementing and revising them, and the results), you are in a good position to use that documentation to reflect on your practice. In this sense, the design narrative, the retrospective account of what you did, why you did it, what you learned  is an essential piece of doing reflective design practice. When you go back and think through your own process you are not simply reporting on what you learned you are actually making sense out of your trajectory and coming to understand what it is that you actually learned. Like much of qualitative and hermeneutic research, the process of writing is not a process of transmission of knowledge but of the discovery of knowledge. Writing a design narrative is the process by which we come to know and learn from our work.

Making Tacit Practical Design Knowledge Explicit and Available: It is essential that the knowledge developed in the design process is documented and shared. While the individual studies that come out of a design research process provide evidence of the value, or of particular lessons learned in part of a design project, they leave a considerable amount of the bigger picture knowledge off the table. Quite frankly, much of the most essential parts of design are not about explaining that something works, if someone wants to get into design they need access to the deeply pragmatic, heuristic driven, knowledge that develops on over time in the process of design. The design narrative is an essential medium for capturing and disseminating this kind of tacit knowledge.

In short, I would suggest that this particular piece of scholarship serves as a great example of the value of reporting design narratives and an exemplar for others to use as a model for composing their own design narratives.

Debating the Digital Humanities Gets Real

My author copies of Debating the Digital Humanities came in today. It’s humbling to have some of my words included in such a hefty tome. I’ve been reading and enjoying it, great stuff. Beyond being a useful volume, it’s also neat to see it incorporate a selection of blog posts. The format of the book is itself an argument for how publish-then-filter can work for humanities scholarship.

It is fun and weird to see things I hadn’t intended for print in print. They have a different kind of materiality to them now. As my words ended up in two of the publish-then-filter parts of the book, I thought I might be slightly interesting to take a moment to reflect on how what I wrote ended up making its way in there.

Blogging about Course Blogging Goes to Print

I teach a digital history course at American University, this is my second time around at the course. After teaching my first incarnation of the course I wrote a series of reflective blog posts about the experience. The goal of those posts was to distill and refine my thinking about the role that public blogging can play as an instructional tool. It is particularly pertinent to the digital history course as participating in online public dialog is a core goal of the course. I was both excited and flattered when Matt asked if I would be game for including one of my posts on the course for the book. See below.

It is fun and neat to have a post end up in a book, but it is also a bit disorienting. On my blog it was part of a threaded run of posts about my teaching and writing. I like to think that everything I write here always remains a draft. Everything I write here is something I might return to and revise. Undoubtedly there will be typos in this post that someone will point out that I will fix. But now, reading the post on paper in this volume, it feels completely different. Instead of being my informal thinking out loud on my teaching it has become something much more enduring. Just look at those type faces! Such dignified serifs. It’s no longer some guys words on the internet. It’s a stake in the ground about the place of technology in teaching and learning in an emerging field. I love it.

Seeing the post in print helps further validate the point of the post and blogging in the course.  It is one thing to stand up in front of a class of students and say, “hey, this blogging thing is important. It changes how power and publishing works. So take it serious, write good stuff and write it in public so you can claim credit.” It’s something completely different to be able to say, “Oh, and when you do blog, sometimes you say something interesting enough that it warrant’s being included in a really cool book.” When I tell my students about this next Wednesday I will have gone from course, to reflection, to book, to this blog post and back to course in seven months. I for one think that is rather neat.

Day of Digital Humanities Definitions

I have one other small contribution in the book. At the end of the first section are a selection of definitions of the digital humanities that some of us provided for the Day of Digital Humanities. See mine below, again in print, in the book.

What’s funny about this is that it’s a flippant comment, a personal aside. Here is some context. When you sign up to do the Day of Digital Humanities you fill out a web form, more or less a registration form. On the form there was a text box to fill in with your definition. It didn’t say “think about this really carefully, because it might end up in a big thick book.” So what I filled in was just what came off the top of my head at the time. To this end, it is all the more jarring to read something I had to fill in on a registration form printed like this. Jarring in a good way. I’m relatively happy with my definition. I’ll stand behind these jottings. Some of the value in these definitions is that they are not diplomatic. They are the things we had on hand at the moment and there is something that is a bit more direct and honest about those kinds of comments.

Trying to do the Digital Humanities Face

In conclusion, here is my best attempt at doing the debates in the digital humanities face. I should probably have shaved before taking the picture, but there lies the perils of just being able to hit the publish button before anyone else intervenes to stop you.

Defining Data for Humanists: Text, Artifact, Information or Evidence?

Fred and I got some fantastic comments on our Hermeneutics of Data and Historical Writing paper through the Writing History in the Digital Age open peer review. We are currently working on revising the manuscript. At this point I have worked on a range of book chapters and articles and I can say that doing this chapter has been a real pleasure. I thought the open review process went great and working with a coauthor has also been great. Both are things that don’t happen that much in the humanities. I think the work is much stronger for Fred and I having pooled our forces to put this together. Now, one the comments we got sent me on another tangent. One that is too big of a thing to shoe horn into the revised paper.

On the Relationship Between Data and Evidence

We were asked to clarify what we saw as the difference between data and evidence. We will help to clarify this in the paper, but it has also sparked a much longer conversation in my mind that I wanted to share here and invite comments on. As I said, this is too big of a can of worms to fit into that paper, but I wanted to take a few moments to sketch this out and see what others think about it.

What Data Is to a Humanist?

I think we have a few different ways to think about what data actually is to a humanist. I feel like thinking about this and being reflexive about what we do with data is a really important thing to engage in and here is my first pass at some tools for thought about data for humanists. First, as constructed things data are a species of artifact. Second, as authored objects created for particular audiences, data can be interpreted as texts. Third, as computer processable information data can be computed in a whole host of ways to generate novel artifacts and texts which themselves open to interpretation and analysis. This gets us to evidence. Each of these approaches, data as text, artifact, and processable information, allow one to produce/uncover evidence that can support particular claims and arguments. I would suggest that data is not a kind of evidence but is a thing in which evidence can be found.

Data are Constructed Artifacts

Data is always manufactured. It is created. More specifically, data sets are always, at least indirectly, created by people. In this sense, the idea of “raw data” is a bit misleading. The production of a data set requires a set of assumptions about what is to be collected, how it is to be collected, how it is to be encoded. Each of those decisions is itself of potential interest for analysis.

In the sciences, there are some agreed upon stances on what assumptions are OK and given those assumptions a set of statistical tests exist for helping ensure the validity of interpretations. These kinds of statistical instruments are also great tools for humanists to use. However, they are not the only way to look at data. For example, most of the statistics one is likely to learn have to do with attempting to make generalizations from a sample of things to a bigger population. Now, if you don’t want to generalize, if you want to instead get into the gritty details of a particular individual set of data, you probably shouldn’t use statistical tests that are intended to see if trends in a sample are trends in some larger population.

Data are Interpretable Texts

As a species of human made artifact, we can think of datasets as having the characteristics of texts. Data is created for an audience. Humanists can, and should interpret data as an authored work and the intentions of the author are worth consideration and exploration. At the same time, the audience of data is also relevant, it is worth thinking about how a given set of data is actually used, understood and how data is interpreted by audiences that it makes its way to. That could well include audiences of other scientists, the general public, government officials, etc. In light of this, one can take a reader response theory approach to data.

Data are Processable Information

Data can be processed by computers. We can visualize it. We can manipulate it. We can pivot and change our perspective on it. Doing so can help us see things differently. You can process data in a stats package like R to run a range of statistical tests, you can do like Mark Sample and use N+7 on a text. In both cases, you can process information, numerical or textual information, to change your frame of understanding a particular set of data.

Data can Hold Evidentiary Value

As a species of human artifact, as a cultural object, as a kind of text, and as processable information data is open to a range of hermeneutic processes of interpretation. In much the same way that encoding a text is an interpretive act creating, manipulating, transferring, exploring and otherwise making use of a data set is also an interpretive act. In this case, data as an artifact or a text can be thought of as having the same potential evidentiary value of any kind of artifact. That is, analysis, interpretation, exploration and engagement with data can allow one to uncover information, facts, figures, perspectives, meanings, and traces which can be deployed as evidence to support all manner of claims and arguments. I would suggest that data is not a kind of evidence; it is a potential source of information which could hold evidentiary value.