Tag: digital preservation

This has been brewing for a while, but it’s now enough of a thing that I can share about it. I am excited to announce that I’m on the hook with Johns Hopkins University Press to produce a short book (30-40k words) called The Theory & Craft of Digital Preservation: An Introduction.

I have about half of the book together in a really rough draft form. Much of my nights and weekends for about the next six months will be spent working up the rest of it and getting the whole thing together.

The genesis of the book came when I was designing my digital preservation seminar and realized that I feel like much of the beaten path for talking about digital preservation has more to do with how we got to what we do now than how it would make sense to explain the issues and topics to folks from scratch. So the course has given me a chance to try out the road-map for the book.

I’ve gotten the OK to share drafts of the chapters as they start to come together. I’ve found that I benefit dramatically from doing my writing in the open where folks can help me refine and sharpen my ideas before they end up fixed in any particular medium.

To that end, I figured I would share most of the book proposal I worked up. In working on drafting, some of this has started to shake out a bit differently, but I thought folks might be interested in a preview. I’m thinking I will start posting a chapter or two a month early-ish in the new year.

Overview of the Book

The historical record is increasingly digital. Over the last half century, under headings of “electronic records management” and “digital preservation,” librarians, archivists, and curators have established practices to ensure that our digital scientific, social and cultural record will be available to scholars and researchers into the future. This book is intended as a point of entry into that theory and practice.

Through years of leading collaborative national digital strategy efforts to ensure long-term access to digital content, I have observed that many experts in digital media and libraries, archives and museums often end up talking past each other as they work toward their mutual goals. All too often, discussions of digital preservation fail to fully state and engage with the nature digital objects and media, thereby undermining our ability to fully engage do this work in a common and coherent fashion.

This failure of understanding is rooted in two key fundamental issues: First, that preservation itself is not a single area of activity, but has always been historically intertwined with distinct disciplines that have grappled with the affordances of various historically “new” mediums. Second, that there are distinct affordances of digital media that require rethinking those diverse perspectives on preservation and conservation. The central contribution of this book is to put the lineages of preservation in dialog with the affordances of digital media as basis to articulate a theory and craft of digital preservation.

As a guidebook and an introduction, this text is a synthesis of extensive reading, research, writing, and speaking on the subject of digital preservation. It is grounded in my work on digital preservation at the Library of Congress and before that, working on digital humanities projects at the Center for History and New Media at George Mason University.  The first section of the book synthesizes work on the history of preservation in a range of areas (archives, manuscripts, recorded sound, etc.) and sets that history in dialog with work in new media studies, platform studies, and media archeology. The later chapters build from this theoretical framework as a basis for an iterative process for the practice of doing digital preservation.

This book serves as both a basic introduction to the issues and practices of digital preservation and a theoretical framework for deliberately and intentionally approaching digital preservation as a field with multiple lineages.  The intended audience is current and emerging library, archive, and museum professionals as well as the scholars and researchers who interface with these fields. As such, the book will be useful as assigned reading for graduate courses in digital preservation and digital curation in library science, museum studies, and public history programs. This book is also highly relevant to digital humanities programs and courses as the work of digital humanists increasingly results in the development of digital platforms, tools and resources which face significant sustainability challenges and thus require an understanding of digital preservation planning to succeed.

There are a handful of books on digital preservation, but this book is significantly different in two key ways. First, it is intentionally brief. Because of this, it is more accessible and usable by a wide range of stakeholders in digital preservation. This is not to an exhaustive work on the subject, but a clear and focused perspective and approach. Second, it treats digital preservation as a craft and anchors it in work in humanities scholarship on media and mediums. Much of the extent work on digital preservation approaches the subject as one that is highly technical, which continues to obfuscate many key issues and assumptions, particularly for humanities scholars interested in understanding digital preservation. While the book has a practical bent, it is not a how-to book that would quickly become outdated. It establishes and offers stages and processes for doing digital preservation, but it is not tied to particular tools, methods, or techniques. Instead, it is anchored in an understanding of the traditions of preservation and the nature of digital objects and media.

Sections of the Book 

Introduction: Getting Beyond Digital Hyperbole

At a summit on digital preservation at the U.S. Library of Congress in the early 2000s, a participant from a technology company proposed, “Why don’t we just hoover it all up and shoot it into space.” The “it” in this case being any and all historically significant digital content. Many participants laughed, but it wasn’t intended as a joke. Many have, and continue to seek similar “moon-shots,” singular technical solutions to the problem of enduring access to digital information.

More than a decade later, we find ourselves amid the same set of stories we have heard for at least thirty years. Among the public, there is a persistent belief that if something is on the Internet, it will be around forever.  At the same time, warnings of a potential impending “digital dark age,” where records of the recent past become completely lost or inaccessible appear with regular frequency in the popular press as well.

To many, it seems like the world needs someone to design a system that can “solve” the problem of digital preservation. The wisdom of the cohort of digital preservation practitioners in libraries, archives, and museums who have been doing this work for half a century suggests this is an illusory dream not worth chasing. Working to ensure long-term access to digital information is not a problem for a tool to solve. It is a complex field with a significant ethical dimension. It is a vocation.

The purpose of this book is to offer a path for getting beyond the hyperbole and the anxiety of the digital and establish a baseline for practice in this field. To do this, one needs to first unpack what we mean by preservation. It is then critical to establish a basic knowledge of the nature of digital media and digital information. With these in hand, anyone can make significant and practical advances toward mitigating the most pressing risks of digital loss. For more than half a century, librarians, archivists, and curators have been establishing practices and approaches to ensure long-term access to digital information. Building from this work, this book provides both a sound theoretical basis for digital preservation and a well-grounded approach to its practices and craft.

Section One: Historicizing Preservation and Digital Media

Chapter One: Preservation’s Divergent Lineages

Interdisciplinary dialog about digital preservation often breaks down when an individual begins to protest “but that’s not preservation.” Preservation means a lot of different things in different contexts. Each of those contexts has a history. Those histories are tied up in the changing nature of the mediums and objects for which each conception of preservation and conservation was developed. All to often, discussions of digital preservation start by contrasting digital media to analog media.  This contrast forces a series of false dichotomies. Understanding a bit about the divergent lineages of preservation helps to establish the range of competing notions at play in defining what is and isn’t preservation.

Building on work in media archeology, this chapter establishes that digital media and digital information should not be understood as a rupture with an analog past, Instead, digital media should be understood as part of a continual process of remediation embedded in the development of a range of new mediums which afford distinct communication and preservation potential. Understanding these contexts and meanings of preservation establishes a vocabulary to articulate what aspects of an object must persist into the future for a given preservation intent.

To this end, this chapter provides an overview of many of these lineages. This includes; the culture of scribes and the manuscript tradition; the bureaucracy and the development of archival theory for arranging archives and publishing records; the differences between taxidermy and insect collecting in natural history collections and living collections like butterfly gardens and zoos; the development of historic preservation of the built environment; the advent of recorded sound technology and the development of oral history; and the development of photography, microfilming and preservation reformatting. Each episode and tradition offers a mental model to consider deploy for different contexts in digital preservation.

The purpose here is not a detailed history of lineages of preservation and the development of media, but instead to illustrate the many different conceptions of preservation exist and how those conceptions are anchored in different objectives. This overview provides readers with a focus on the distinct conceptions of what matters about an object and the innate material properties and affordances of different kinds of media as they relate to preservation.

Chapter Two: Understanding Digital Objects

Doing digital preservation requires a foundational understanding of the structure and nature of digital information and media. This chapter works to provide such a background through three related strands of new media studies scholarship. First, all digital information is material. Second, digital information is best understood as existing in and through a nested set of platforms. Third, that the database is an essential media form and metaphor for understanding the logic of digital media.

Given that digital information is always physically encoded on digital media, it is critical to recognize that the raw bit stream (the sequence of ones and zeros encoded on the original medium) have a tangible and objective ability to be recorded and copied. This provides an essential first level basis for digital preservation. It is possible to establish what the entire sequence of bits is on a given medium, or in a given file, and use techniques to create a kind of digital fingerprint for it that can then be used to verify and authenticate perfect copies.

With that noted, those bit streams are animated, rendered, and made usable through nested layers of platforms. In interacting with a digital object, computing devices interact with the structures of file systems, file formats and various additional layers of software, protocols and drivers. Drawing on examples from net art, video games, and born digital drafts of literary works, I explore multiple ways to approach them anchored in different layers of their digital platforms. The experience of the performance of an object on a particular screen, like playing a video game or reading a document, can itself obfuscate many of the important aspects of digital objects that are interesting and important but much less readily visible, like how the rules of a video game actually function or deleted text in a document which still exists but isn’t rendered on the screen.

As a result of this nested platform nature, the boundaries of digital objects are often completely dependent on what layer one considers to be the most significant for a given purpose. In this context, digital form and format must be understood as existing as a kind of content. Across these platform layers digital objects are always a multiplicity of things. For example, an Atari video game is a tangible object you can hold, a binary sequence of information encoded on that medium identical to all the other copies of that game, source code authored as a creative work, a packaged commodity sold and marketed to an audience, and a signifier of a particular historical moment. Each of these objects can coexist in the platform layers of a tangible object, but depending on which is significant for a particular purpose one should develop a different preservation approach.

Lastly, where the index or the codex can provide a valuable metaphor for the order and structure of a book, new media studies scholarship has suggested that the database is and should be approached as the foundational metaphor for digital media. From this perspective, there is no “first row” in a database, but instead the presentation and sorting of digital information is based on the query posed to the data. Given that libraries and archives have long based their conceptions of order on properties of books and paper, embracing this database logic will have significant implications for making digital material available for the long term.

Chapter Three: Challenges  & Opportunities of Digital Preservation

With an understanding of digital media and some context on various lineages of preservation, it is now possible to break down what the inherent challenges, opportunities and assumptions of digital preservation are.

We can’t count on long-lived media, interfaces, or formats. Popular digital media of all kinds Disc, Disk, and NAND Flash Wafers all degrade rather quickly — in terms of years, not decades or centuries. Many of these media are relatively complex to read, so the interfaces required to interpret them are likely to not be particularly long lived. The costs of trying to either repair these media or to fix and repair interfaces to read them rapidly becomes prohibitive. As a result, traditional notions of conservation science are, outside of some niche cases, going to be effectively useless for the long-term preservation of digital objects.

Going back to the discussions of preservation lineages, this means that digital preservation is an enterprise that can only focus on the allographic digital object. While all digital information is material, the conservation of that material over the long haul is not broadly practical. Where conservation science is concerned with the chemical and material properties of mediums and artifacts, the science of digital preservation is and will be computer science. With that said, because bitstreams are always originally encoded on tangible media and then created by, acted on and interpreted by all kinds of human made layers of software they end up presenting an extensive range of seemingly artifactual and not simply informational qualities. That is, the physical and material affordances of different digital mediums will continue to shape and structure digital content long after it has been transferred and migrated to new mediums.

Section Two: Doing Digital Preservation

Chapter Four: Articulating preservation intent

What is it about the thing you want to preserve that matters and what do you need to do to make sure it is there in the future? To many, this seems like a simple question. It is not. Too often we take for granted that there is a de facto answer to this question. However, as a result of the nested platform nature of digital information and the fact that most of what we care about is the meaning that can be made from collections of objects, it is critical to be deliberate about how we answer this question in any given situation. This is why digital preservation must be continually grounded in the articulation of preservation intent.

In some cases, someone can clearly articulate this intent at the start of a project. But  for most preservation projects it is often best to be purposeful and strategic around the preservation intention. This is particularly critical given that deciding what matters most about some set of material can lead to radically different approaches to preserving and describing it.

Through examples of the diverse types of content that different kinds of cultural heritage organizations are preserving and their intent for doing so, this chapter establishes how to articulate preservation intent and how well-articulated preservation intent makes the resulting collections easier to evaluate and more transparent for future users.

Chapter Five: From Bit Preservation to Digital Preservation

Taking into account the challenges and opportunities of digital preservation, it is important to bracket the work into two different challenges: bit preservation and digital preservation. Bit preservation, ensuring authentic copies of digital objects, is the most pressing problem. Thankfully, it is a relatively straightforward problem for which there are a range of simple solutions. With that said, ensuring those authentic copies are interpretable, comprehensible and usable is far more challenging. Thankfully, this work of digital preservation is a much less time sensitive activity.

Bit preservation is accomplished by managing multiple copies of the digital objects you want to preserve, regularly comparing digital fingerprints for those files to ensure that they are all identical, repairing or replacing copies when they fail those checks, and migrating the copies to newer media and continuing to ensure that the digital fingerprints still match. With more resources, there are better ways to systematize and automate these processes, but with relatively small collections it is still possible to do this and be confident you have authentic copies as long as someone continues to mind and tend to them.

Digital preservation is much less straightforward.  The central challenge of digital preservation is that software runs. The active and performative nature of that running is only possible through a regression of dependencies on different pieces of software that are typically tightly coupled with specific pieces of hardware. Along with this, it is important to think through if there is enough context for the digital objects for someone in the future to be able to make sense of them. Two primary strategies exist for approaching these issues: emulation and format migration. Both are discussed and a case is made for why in many cases organizations are hedging their bets and pursuing both strategies.

Chapter Six: Arranging and Describing Digital Objects

The story goes that shortly after the Library of Congress signed an agreement with Twitter to begin archiving all of the tweets, a cataloger asked “But who will catalog all those tweets?” The idea of describing billions of objects was dauntingly incompressible to those who lacked experience with the nature of digital media. Like most digital objects, tweets come with a massive amount of transactional metadata: timestamps, usernames, unique identifiers, links out to URLs on the web. Like most digital objects, the tweets can largely describe themselves.

The usability of digital information will be largely dependent on how we organize, arrange, and describe it.  Arranging and describing digital objects needs to conceptually shift to embrace the nature of digital media and to recognize a distinct transition which has occurred in terms of computability. Digital media continually generates massive amounts of metadata and because it is computable, it is also increasingly possible to process digital data to derive descriptive information and metadata. As a result, arranging and describing digital content should increasingly be focused on limited amounts of expert intervention in chunking and describing content in aggregate and leaving lower levels of description to the objects themselves.

In terms of arranging digital objects, their database nature means that unlike folders in a box or books on a shelf, by their very nature digital media come with a multiplicity of orders. This complicates core archival principles around original order. It also, requires thinking through how to chunk content into reasonable and coherent sets of information that are easier to manipulate and work with as all kinds of current and future users.

In this context, it is critical to revisit the levels of description at which librarians, archivists, and curators work to evaluate in what cases something should be treated as an “item” or a “collection” and what levels of descriptive work should be employed. Given how much objects are self- describing, it makes much more sense to take up archival practices of describing content at the collection level and explaining the scope of a collection, the context of it’s acquisition, and how and why that collection was collected and preserved and to let the lower levels of description be left to the content itself.

Similarly, many digital objects actually index, describe, and annotate other digital objects. For instance, if you take all of the links that appear in articles published in the Drudge Report, the fact that the Drudge Report linked out to those sites tells you something about them. This affords the possibility of starting to think of nearly all-digital objects as both data in their own right and metadata that describes other objects. To this end, we must increasingly think of “description” and “the described” as a fuzzy boundary.

Chapter Seven: Divergent and Multimodal Access and Use

When a user in a research library asks to see a book in an obscure language a librarian will generally bring it out and let them look at it. That librarian may have no idea how to make sense of the text, but they know how to provide access to it and it is assumed that the researcher needs to come with the skills to make sense of it. At the most basic level, we can provide this kind of access to any digital objects we are preserving.

The affordances of digital media open up significant potential for access and use of digital content. At the same time, our experience with commercial software can get in the way of letting others access digital content until one can provide a simple way for any user to double click on a digital object and have it “just work.” It is critical for us to get over the assumptions that are embedded in this mentality and embrace the divergent and multimodal nature of access that digital media present us with.

This means digital preservation practitioners need to be OK with just saying, “Here it is, have at it” and also with consistently exploring the potential for new tools and methods for providing access to digital content. Even if you don’t know how to open a given file, there are a range of emerging techniques and approaches that researchers today and in the future will be able to use in working with digital content. In addition, it is important to think through the types of access restrictions or redaction of information may be necessary.

This means we should be continually exploring ways to make digital content as broadly accessible and usable as individual files, bulk aggregates and a range of other modes. Researchers are increasingly interested in approaching all kinds of digital content as data sets for computational analysis and this requires adopting new ways of thinking about access.

Conclusions: The Theory & Craft of Digital Preservation

Digital preservation is not an exact science. It is a craft in which experts must reflexively deploy and refine their judgment to appraise digital content and implement strategies that make the most sense for minimizing the most pressing risks of loss while working to make it as widely usable and useful as it can be to its’ respective audiences. At least, that is the case I have sought to make in this book. As Stacy Eardman, digital archivist at Beloit College has noted, digital preservation is much like a lyric from the song The Have Nots, “This is the game that moves as you play.”

The craft of digital preservation is anchored in the past. It builds off of the records, files, and works of those who came before us and those who designed and set up the systems that enable the creation, transmission and rendering of their work. At the same time, the craft of digital preservation is also the work of a futurist. We must look to the past trends in the ebb and flow of the development of digital media and hedge our bets on how digital technologies of the future will play out.

My former supervisor, Martha Anderson, who worked as the Managing Director of the National Digital Information Infrastructure and Preservation Program at the Library of Congress, liked to describe digital preservation as a relay race. Digital preservation is not about a particular system, or a series of preservation actions. It is about preparing content and collections for hand offs. We cannot predict what future digital mediums and interfaces will be, or how they will work, but we can select materials from today, articulate aspects of them that matter for particular use cases, make perfect copies of them, and then work to hedge our bets on digital technology trends to try and make the next hand off as smoothly as possible.