The Importance of Archiving and Preservation

Archiving and preservation are central to the long-term success of Perseus as well as the vitality of the discipline of the humanities. A brief examinination of the Perseus Digital Library will underscore the importance of preserving both the data and behaviors implicit in some of its user interfaces. The paper will then focus on three of Perseus' current and future options for archiving both data and associated behaviors: being open, distributing for redundancy, and leveraging institutions. In applying each of these principles, not only are issues of preservation addressed, but accessibility and innovation within the humanities as a whole is increased.

One of the newest tools Perseus has developed is the tiled image viewer. Using this interface, built upon the Google Maps API, users are free to explore Perseus' high-resolution images by zooming in up to five levels and/or panning back and forth to view the image in full. Currently, the tiled image viewer is in beta and exposes Perseus' coin images from the Dewing collection (numbers 1990.26.0001 - 1990.26.0499). In addition, images of the Comparetti, a Homeric manuscript photgraphed by Harvard's Center for Hellenic Studies, are also exposed. (Example 1, Example 2).

Looking closely at Example 1 from the Comparetti, one can see the main text of the Iliad, illustrations, and a wealth of notes. The text's visual richness reflects a semantic richness that makes it extremely valueable to this day. Just as the Comparetti manuscript uses the sze and position of text to describe numerous associations between primary and secondary texts, Perseus' HTML text reader makes use of similar visual conventions for illustrating these relations. Like the Comparetti, Perseus' reader displays passages from alternative editions alongside the main text. However, Perseus uses technology to go a step further by providing a lists of all passages within the digital library which reference the text, and extracted named entities like people, places, and organizations, occurring within the passage. Furthermore, for Ancient Greek and Latin passages, the user may click on any word and retrieve possible meanings.

The Comparetti illustrates the importance of archiving and preservation. In the past, to view teh Comparetti, one would have to travel to Italy and gain access. However, technology now makes it possible to simultaneously increase both the liklihood of survival and accessibility to such manuscripts. Access tot he Comparetti is literally a click away. Now people can do more than just view the Comparetti, they can zoom in to read a character which isn't clear or zoom out to view the overall structure of the page. The Perseus reader further increases accessibility by lowering the language barrier. Using the reader, the definition of an Ancient Greek, Latin, and soon Arabic word is just a click away. These interfaces do more than just present data, they define mechanisms for interacting with it. Such mechanisms are not limited to the user interface, but can be defined for the service layer as well. This is precisely what CTS does for texts. For Perseus, it is no longer adequate to preserve the data, the behaviors defining interactions with the data also must be preserved.

Perseus' three current options for helping its data and behaviors to survive include being open, leveraging distribution, and relying upon institutions. These options are not mutually exclusive and each of these principles can be applied to data and behaviors. The first preservation principle, being open, lies at the center of Perseus' strategy for long term preservation. For data, being open means using formats which are application independent. A proprietary file format depends upon an institution or a company which can go out of existance. To increase the probability of survival of the data, it must be formatted in a manner that can be read by many different types of programs and platforms. Furthermore, the data should be stored in a manner so that it is easily transformable. Currently, Perseus uses XML, allowing data to easily be transformed into any format desired. Perseus' history illustrates the importance of easily transformable, open data formats. The XML texts in use today originally were SGML documents. The open nature of SGML enabled Perseus to view and transform the data into XML as technology evolved. Finally, open data means that other people are free to use it. By licensing texts under the Creative Commons license, not only does Perseus increase their survivability, but it increases access to the data. Since people who access the data are free to create derivative works, Perseus increases the opportunities for innovation within the field of the humanities.

Open behaviors also play a crucial role in long term preservation. The Canonical Text Services exemplify this principle because they are a protocol specification, but also have a working implementation expressed as an open source API. A protocol specification details both the semantics and syntax for interacting with data, independent or loosly-dependent upon technology. Ideally, behaviors defined by the specification should transcend technology while providing a consistent interface for interacting with the data. For example, CTS' requests might be implemented using a variety of mechanisms, perhaps its a series of XSLT transformations used in a Cocoon pipeline, or perhaps its a war deployed in a java servlet container. Regardless of the technology used, one can interact with the data in the same way, and one knows the meaning of these interactions through the semantics detailed in the protocol specification. Protocol specifications are essential to scholarly work since they formally express the meaning of the behaviors used. When actually using CTS however, people are free to download an implementation, modify it, and explore how it works. In the best case, implementations define an API which others can use. APIs encode domain knowledge and by allowing others to download them, more people think in terms of that model of domain knowledge. APIs have the added benefit of lending themselves to quantitatively measuring the degree to which they comply to a specification through unit testing. Furthermore, people can see exactly how a CTS request works, making it easier to implement using a future technology and increasing the opportunity for that behavior's survival into the future. Just as open data had the added benefit of increasing innovation within the humanities, so do open behaviors have the additional benefit of enabling rigorous scholarship to occur. If someone uses a behavior to support a thesis, another researcher can examine exactly how that behavior works and decide if the thesis still holds. Not everyone will choose to look this closely to determine the merit of an argument, but to not be able to look at the code, when desired, is analogous to asking a mathematician to believe a theorem without seeing a proof. Every step of the rational argument must be exposed for true rigorous scholarship to occur. Open behaviors enable the use of technology in the rational thought process by formally defining their meaning while allowing others to examine their inner workings. For a scholarly institution like Perseus, behaviors must be open if the rational processes they represent are to be preserved.

In using open data and behaviors, Perseus is free to fully leverage distribution as a preservation mechanism. Currently, Perseus is pursuing two strategies for preserving its data through distribution. The first strategy for distribution is to give away as many copies of its raw data as possible. Perseus can do this because it has licensed it data through the Creative Commons license. The second strategy involves using technologies such as SRB and iRods to transparently distribute data on a grid while appearing to be on one logical filesystem. Distribution of data greatly increases the chances of its survival, as demonstrated by the Library of Alexandria. The Library of Alexandria had a wealth of information which was lost due to fires. This archiving failure was not due to the incompetence of the librarians of Alexandria, but a reflection of the technology during that time. To create a copy of a text, hand copying was the only option. Because of this, the likelihood of survival of data was tied to a few physical artifacts. Today, technology enables us to make one or one billion copies by typing a single command, rather than writing billions of words by hand. With such technology available, it is the duty of content providers like Perseus to make and distribute copies of their data so that future generations might also marvel at works like the Comparetti and enhance them using the technologies of their generation.

Perseus applies geographic distribution to its behaviors as well. One of the most basic ways this is accomplished is through mirror sites. Over Perseus' history, Perseus has been hosted at Berlin, and the University of Chicago, in addition to Tufts. If anything happens to one of these sites, users are still able to access Perseus' services from another server. A side effect of using mirrors is that Perseus' codebase then becomes geographically distributed, producing more copies of this resource, increasing the chances of its survival into the future. A second technology that may enable the geographic distribution of Perseus' behaviors is grid technology. Not only would the grid lead to an increased gain in performance, but it would transparently distribute these behaviors geographically under one logical interface. Perseus is in a position to distribute its code because it is open source, underscoring yet again the importance of being open with behaviors for long term preservation and increased short term accessibility.

Finally, Perseus relies upon institutional repository as the third component of its preservation strategy. In the context of data, institutions have policies for ingest that ensure a certain level of quality. Institutions provide a nice complement to the distribution strategy. The Library of Alexandria had immense amounts of quality content. Similarly, institutions have policies that ensure a certain level of quality of content and an expertise in archiving and preserving data. When combined with distribution, institutional repositories allow Perseus to leverage this expertise and quality, without having to worry about putting all of its eggs in one basket.

Institutional repositories using frameworks such as Fedora also enable Perseus to preserve behaviors. The process of moving Perseus' services into an institutional repository has forced more documentation of the behaviors which they provide. Documentation, of both the specification and implementation has been improved as a result of working with Tufts' Digital Collections And Archives (DCA). If the DCA chooses to use the same implementation of a behavior as Perseus, then the documentation of the implementation, as well as the implementation itself may be improved. If the DCA decides to use a different implementation, then the extent to which the specification is truly independent of the implementation is tested. In going through this process, Perseus ensures that its behaviors are well defined in a manner that transcends the current technology, and can be maintained over time depending upon the technology used.

Being open, distributing for redundancy, and leveraging instutitions form the three components of Perseus' long term preservation strategy. Through applying these principles, Perseus greatly increases the chances of long term survival of data and associated behavior while fostering accessibility and innovation within the field. In a time when technology makes it easier than ever before to preserve this data for generations to come, it is the duty of content providers like Perseus to apply technology to this end. In doing so, Perseus does its part in ensuring that future generations might also enjoy editions like the Comparetti.

Adding Value to Open Scholarly Content

One of the ways in which the Perseus Digital Library increases accessibility to and interest in the humanities is through making its content freely available. Perseus can give away its content and still keep its users coming back because it provides semantically precise associations between content. The Canonical Text Services (CTS) protocol and the CTS-URN syntax it defines exemplify the types of services Perseus uses to intra and inter-connect content. However text services are just one aspect of Perseus' service layer, a layer which represents just one-quarter of Perseus' overall logical architecture. Each of the layers of Perseus' logical architecture increase the value of the humanities and thereby the value of Perseus in the eyes of its users.

To understand how Perseus can give away its texts without losing users, it is helpful to distinguish between Perseus' static and dynamic content. Perseus' TEI-XML texts, artifact and image metadata, and named-entity and morphological datasets are static data that Perseus currently or will distribute freely under the Creative Commons license. Perseus can afford to do this because its value lies in the associations it can create by making its data dynamically accessible. These semantically precise associations within Perseus' own content (intra-connecting) and between its own content and external data and/or services (inter-connecting) give users a wealth of context for understanding and interpreting Classical data.

The CTS protocol will make Perseus texts dynamically accessible, allowing them to be connected in ways that downloading a bunch of TEI-XML texts simply does not provide. The Canonical Text Services protocol uses URNs to reference texts in terms of their hierarchical structure. A cousin of the Fundamental Records For Bibliographic Records (FRBR), CTS URNs may identify the work of an author or an edition or translation of a work, but extend FRBR by referencing texts by their logical citation scheme. Rather than navigating by page number, CTS interprets the semantics of these URNs to retrieve logical sections of a text organized by chapter, section, or some other scheme. Furthermore, the protocol specifies extensions to the CTS URN for referencing an arbitrary sequence of characters within a passage, providing a syntax for textual alignment across editions without losing context.

CTS URNs enable Perseus to create associations which increase the value of its data. Just as Google Page Rank takes HTML links into consideration to measure the value of a page to a user, Perseus can increase the value of its content to its users with far greater semantic clarity and precision by using CTS URNs. Some services that will eventually be exposed via index services include Perseus' named-entity disambiguation, citations, and morphological information. For each of these examples, CTS-URNs and collection IDs will be combined with Index Services to make connections within Perseus' own data. However, URNs also allow Perseus to connect its highly-structured texts with external data and services; one such service is searching using Google Base.

Connecting highly-structured data with less structured data through search is not unfamiliar. When searching in Google Earth, the display window represents a range of geographic coordinates currently visible to the user. When a user performs a search, hits are displayed in terms of their geographic coordinates as markers. A CTS-aware search would be similar, results are limited to the range of textual coordinates associated with the current passage and displayed in terms of their textual coordinates. Just as Google Earth parses KML documents and provides services consistent with the seamntics of longitude and latitude, so could a CTS-enabled search parse CTS URNs, interpreting the semantics of this coordinate system for textual reference. Currently, Perseus is experimenting with using CTS-URNs within queries in Google Base. After creating an item for each logical section of text and specifying the corresponding URN in the page's metadata, this less-structured data is uploaded to Google Base. Each of these items links to Perseus' text page, which provides rich context for the item. Using this approach, one can search for a text from the author to passage level (it would be very difficult to generate an item for each character on a page) and get relevant hits that point directly to Perseus. Furthermore, a URN specified in combination with a term, such as 'horse' provides a mechanism for limiting search results to the text(s) represented by that URN. (Experimental Examples: Caesar's The Gallic War, a Perseus edition English translation of Caesar's The Gallic War, Book 1, Chapter 1 of the aforementioned edition, occurrences of 'horse' within the aforementioned edition)

CTS provides numerous benefits not inherent in the raw data, but which are emergent through the behaviors it defines. The CTS protocol defines a standard mechanism for referencing and retrieving texts. Since it is an open protocol, with a functioning implementation exposed via an API, the semantics and behaviors of CTS are specified independently of implementation, and are possible to implement. The specification allows one to quantitatively measure how effective a given implementation is by unit testing the its conditions. The implementation provides a well-defined API that encodes the domain knowledge resulting from working with and a desire to reference texts independently of representation whether a manuscript, book, pdf, or web page. Since all requests to CTS require a CTS-URN, tracking users in a semantically-meaningful way becomes possible as the URN will appear in HTTP access logs. Questions such as how users are navigating the text, what services are being invoked on a given reference, and what requests were performed on that reference can all be answered in terms of the underlying logical structure of the text. Furthermore, since CTS was designed to work with Ancient Greek and Latin texts, it can handle multi-lingual content, and provides a syntax for datasets of aligned texts. Logical referencing of text independent of physical representation, a specification with clear meaning, an API whose functionality can be quantitatively verified, and a notation for semantically precise associations illustrate how dynamic services add value to static content and keep users of Perseus coming back.

Although text services are central to Perseus' mission, they are just one of the services Perseus offers and the entire service layer only accounts for one-quarter of Perseus' logical architecture. Perseus' logical architecture can be used to classify Perseus' other sources of value that both increases the value of the humanities and the value of Perseus in the eyes of its users. First, Perseus' data layer, comprised of TEI-XML texts, databases, and other raw data is freely distributed under the Creative Commons license. Not only does this establish Perseus as a data source to the community, but distributing multiple copies of each text increases its chances of surviving into the future. Rather than having to copy a text by hand, digitization provides the humanities with the ability to make and distribute an arbitrary number of copies, increasing the accessibility and survivability of each text. Furthermore, Perseus' expertise in digitizing these texts serves as a source of value for those who wish to create their own digital editions. Second, the domain layer, where behaviors are associated with the raw data encodes the knowledge and experience gained while working with the content. Working in the domain of Classical texts provides Perseus with a unique perspective on the nature of text that others may find useful, and so gives Perseus the opportunity to help others make sense of their content. Third, Perseus' service layer, provides a series of APIs implementing a set of protocols for each of the types of data Perseus serves. Many of these services rely upon the protocols specified in the TICI stack. Through the service layer, others are free to repurpose Perseus' content through an API that encodes domain knowledge and since the API freely available under an open source license, the community using the API becomes a source of information and value as well. Finally, the display layer, whether widgets, HTML pages, or PDFs gives all users a convenient and easy way to access Perseus' data. The user interface reflects the knowledge gained when building the other layers, and so helps the general public visually see the relations between Perseus' data.

Perseus can give away its static data because it adds value through providing semantically rich associations, adding context to its content. The CTS protocol offers a new way to conceive of, reference, and deliver texts and CTS-URNs provide a syntax for specifying relations among Perseus' and external content and services. These relations increase the value of Perseus in a way that is not inherent in the raw data, but comes from creating associations among the data. In giving away its raw data, Perseus encourages others to develop their own associations, increasing its value as a data provider and as service developers while increasing access to and therefore innovation within the humanities.

Credits:
Thanks to John Blossom whose "Shoreviews. Content Industry Outlook 2007: Reality Checks" gave me criteria for evaluating CTS as a value-added service in the context of the publishing community. Thanks to Gregory Crane for his ideas on interconnecting primary and secondary sources within the Perseus Digital Library and his initial recommendation to look at Google Base. Thanks to Neel Smith for the comparison between searching within Google Earth and a CTS-aware search. Thanks to my brother, Michael Weaver, for his work on the logical layers of an application and their relation to business processes.

Notes:
Based upon slides presented on the panel Getting Search Right for Premium Content during the Spring 2007 ASIDIC meeting.