Trevor Owens

How should digital humanities scholars develop research questions? Spurred on by this recent conversation on twitter, I figured I would lay out a few different ways to go about answering this question about questions. The gist of the dialog is that Jason Heppler suggested that one should “Fit the tool to the question, not the other way around” in terms of working with various kinds of new digital humanities tools. I take tools here to mean any computational instrument employed to understand the world; for examples GIS, topic modeling, creating simulations using cellular automata or agent based models, analyzing frequencies of audio files, or visualizing trends in images. I get where Jason was going, but at least as it was formulated I don’t think it is the right advice.

The conversation prompted me to try and clarify a bit of how I see the relationship between research questions, primary sources, and tools and methods.

Start with the Question, the Archive or the Tool?

Some historians start with their question, some start with a familiarity with a period that suggests that exploration of a particular archive or collection of primary resources could answer. Here are two examples I can recall from colleagues who I worked with doing research in the history of science.

One colleague was aware of the shift that had occurred between classical and modern physics in one astronomer’s work, documented in a recent essay. So he went to look at the papers of another astronomer, which had not yet been particularly well explored, to see if similar or different responses to the notion of a distinction between classical and modern physics had emerged in that astronomer’s work. In short, it was largely about abstracting the results of one exploration into the information available in another individuals archive.

In either case, it’s a bit of a dance between formation of questions and the ways that those questions open up or shift and change as one gets into the complicated, rich and vast space of the possibilities of primary sources.

The Function of Research Questions in History/the Humanities

Back up a bit. What is the purpose of research questions in the humanities? I would posit that the purpose of them is to clarify what is in and out of scope in a project. To define where a project should start and end. Lastly, research questions provide a constant point of reference to check back on when working on a project. You write down your questions as you go, and you can always pull them out again and check to see if, in fact, you are actually working to answer them or if you have drifted off to some other problem. Research questions are useful structures to organize your work and inquiry and they are valuable tools for signifying to others what to expect from a piece of scholarship. Research question are functionally an attempt to establish the set of criteria by which a piece of scholarship should be evaluated.

The Problem of Research Proposals and Fancy Writing

One of the big problems in talking about research questions is that one often describes research questions and methods in research proposals (for grants or dissertations etc.), and those proposals are often really a form of what Joe Maxwell calls “fancy writing.” That is, those kinds of research proposals are more about the performance of demonstrating how smart you are and why you should be given permission to do work than they are about actually trying to get research done. If you haven’t read it, I can’t recommend Joe’s Qualitative Research Design: An Interactive Approach strongly enough. In focusing on the actual purpose of research design and not the performance of proposal writing he cuts through a bunch of the fancy stuff to get to the way that research questions actually develop and evolve. He calls it an interactive approach, but I think iterative would be just as descriptive.

In Maxwell’s approach, there are five components of research design as it is actually practiced.

1. Your goals (the reason you are doing the research),
2. Your conceptual framework (the literature you are working in, your field, your experience that you draw from),
3. Your research questions (a set of clear statements of exactly what you are studying)
4. Your methods (broadly conceived as the way you are going to answer the question, so for historians both the archives/sources you will work from and their perspectives are relevant as well as the way you will sample/explore them, and the actual techniques you will use to analyze and interpret them)
5. The validity concerns and threats (literally, answers to the question “how might you be wrong” where you work through inherent limitations and biases in your methods, sources, perspective, etc.)

The diagram below illustrates how 5 components of design interact

The main point of the diagram, is that your research questions should be iteratively revised and refined throughout the work based on all the four other things that you are working on.

So… research questions aren’t something you state and then follow through on, they are best thought of as statements about your inquiry that are iteratively refined through the process of defining what you are working on.

Generally, the way that research questions are stated in quantitative research is bogus, or at least, bogus in terms of the way that people who do more qualitative research think of research questions. That is, you do a lot of work and scholarship before you can ever formulate a hypothesis that you can test. In that case, you end up with a research question at the end of an exploration not at the front of it.

Tools, Archives, & Research Questions are Inherently Theory Laden

Getting back to the issue of questions, tools, and sources; being good humanists, it is worth leaning back to grok that all method is theory laden. That is, every attempt to answer a question comes with inherent theoretical assumptions about the problem and limitations in what that method can provide in terms of answers. This is true of method broadly conceived; every method for collecting sources/evidence, the original intent by which records and sources are collected create silences, identifying a problem, interpreting sources, composing and reporting on results, all of that, comes with some inherent biases.

That is, all tools, all archives and all research questions are in and of themselves instrumental. We use them in an attempt to understand the world. That is they all serve as lens like tools reflecting and refracting back information in a tool like fashion. I’ve always liked the way that Umberto Eco explains this in Kant and the Platypus as a core concept in hermeneutics; we make interpretations but the underlying reality of existence exerts the force to resist some of those interpretations by simply saying “No” by making it clear that an interpretation can be refuted. A hermeneutics of data that emerges through the use of tools.

So where to start? Start wherever, as long as where you start is anchored in your goals. The hermeneutics of screwing around is itself invaluable. A technique of messing with tools and datasets at hand may well surface interesting patterns that no one would have found if they were working at sources in a another fashion. Pick and archive and find the questions. Or, just start with your questions and work it that way. Whatever you do, realize that it’s an exploratory process.

What matters most in where you start is your actual goals in doing the research. That is, why is it that you are actually doing your work? What is it that you hope your work will potentially do. Don’t confuse your goals with what you are interested in, realize and recognize that your goals area about the purpose of your work. If you want to do work that ultimately helps to understand and give voice to the voiceless then you likely don’t want to start messing around with the text of inaugural presidential speeches. If you want to figure out new kinds of things that can be done with topic modeling then you would presumably want to start with some sources that are in a form or close to a form that you can topic model.

Thanks to Thomas Padilla and Zach Coble who reviewed and provided input on a draft of this post.