This article was co-authored with generative AI. Facts have been checked against public documentation where feasible, but errors may remain. Please verify primary sources before relying on this for important decisions.
Overview
In a workflow for long-term preservation of born-digital materials, there is a step that records the state of the transferred material before turning the files into a preservation package (SIP / AIP). In OAIS (Open Archival Information System) and in archival practice, this is the stage called accessioning.
DataAccessioner is a desktop tool that, at this accessioning stage, safely transfers files from media or folders while capturing and recording each file's checksum and technical metadata. This article organizes what this tool does, why the timing "before making a SIP" is meaningful, and how its role divides from the SIP-creation tool I am working on.
Its Position in the Workflow
The typical flow of digital preservation is as follows. Accessioning is positioned before arrangement, and DataAccessioner handles this stage.
However, it is not accurate to think of the division of roles as "cleanly split along a timeline." Archivematica does not handle "only" ingest and after; it handles the whole from accessioning and characterization (format identification, checksum assignment) at the Transfer stage → Arrangement → SIP creation → Ingest → AIP. In other words, characterization at accessioning time, arrangement, and SIP creation all happen inside Archivematica, so there are parts that functionally overlap with DataAccessioner. It fits reality better to see the two as divided not by time but by the boundary of custody (before entering the preservation system, or inside the system).
:::message The OAIS SIP (Submission Information Package) is not the deliverable of a single fixed step; it is a concept representing a role: "the package that a Producer hands to the OAIS for Ingest." Depending on where you draw the OAIS boundary (whether you regard Archivematica alone as the OAIS, or the institution's whole pipeline as the OAIS), "what is a SIP" and "who makes it and when" change. Note that OAIS has no "arrangement" step; that is a concept from archival practice, and it overlaps within OAIS's Ingest. :::
In the arrangement step, an archivist rearranges the received set of files into a meaningful hierarchy. At this point, unless you preserve the correspondence between the original folder structure (before) and the rearranged structure (after), you cannot later trace "where was this file originally."
The post-arrangement structure is recorded on the SIP / AIP side. As for the pre-arrangement state, when the downstream is Archivematica, it is retained as each file's originalName (the original path at accessioning) and as the transfer METS bundled into the AIP. On the other hand, when the downstream is heterogeneous or undetermined, or when you need a system-independent accessioning = custody trail not tied to a specific preservation system, it becomes hard to reconstruct later unless you fix it at the accessioning stage. DataAccessioner can be positioned as the tool that handles this accessioning-time snapshot (system-independent fixity and original state).
What DataAccessioner Is
As far as I investigated, it is a tool like the following.
- The first version was created in 2008 at Duke University Archives by Seth Shaw. It was later open-sourced with the support of the Digital POWRR project, and is now maintained at
digitalpowrr/DataAccessioner(with contributions from Seth Shaw, Scott Prater, and others). - Within what is publicly available, the latest is v1.2 (released February 17, 2024).
- It is a Java / Swing GUI application that can transfer files from folders or media to a file server without the command line.
- During transfer, it wraps FITS (File Information Tool Set) to capture format identification and technical metadata for each file.
Its distinctive point is that everything is completed in the GUI; the design follows Digital POWRR's aim that even small institutions with limited resources can create accessioning records. It has a lower barrier than handling FITS directly, distilled into the operation of "select a folder and run."
What It Captures and Outputs
Within the range of documentation and explanations I checked, its behavior is as follows.
- Fixity (identity): It computes each file's MD5 checksum before and after copying, and compares the two to detect transfer errors. MD5 is created for both the original file and the copy destination.
- Metadata extraction: It extracts metadata from each file with FITS and saves the result as XML. On the metadata-management side, by default it converts the FITS output to the PREMIS (Preservation Metadata: Implementation Strategies, v2.2) format. This conversion can be changed by swapping the XSLT.
- Report: It includes items such as the directory path, file name, last-modified timestamp, size (bytes), MD5, and file format. A separately distributed Metadata Transformation tool (an XSLT processor) can convert part of this into a spreadsheet or HTML.
In other words, when you ingest a folder or media, it aggregates "original path + MD5 + technical metadata" per file into a single XML and fixes it as an accessioning record — that is the tool.
Relationship to Adjacent Tools
Looking at the role of capturing and recording identity and metadata at accessioning time, there are several adjacent tools.
| Tool | Role | Notes |
|---|---|---|
| DataAccessioner | Captures technical metadata + MD5 via FITS at accessioning and fixes them in XML | Maintained by Digital POWRR. Java / Swing |
| Exactly (AVP + University of Kentucky Nunn Center) | Establishes provenance/fixity at capture time via BagIt + MD5 manifest | For conceptual reference. Current maintenance and availability unconfirmed |
| Archivematica | Transfer (accessioning, characterization) → Arrangement → SIP creation → Ingest → AIP, generating METS / PREMIS | Encompasses accessioning through AIP. Has some functional overlap with DataAccessioner |
DataAccessioner and Archivematica are divided in role not by a simple timeline but by the boundary of custody. Archivematica also performs accessioning and characterization at the Transfer stage, and handles arrangement and SIP creation inside as well, so there is functional overlap. DataAccessioner's unique value lies in fixing fixity and original state independently of any specific system, before entering the preservation system, at the moment of first contact from the media.
Points to Examine When Considering Adoption
- Within what is publicly available, v1.2 (February 2024) is the latest, and its GitHub stars are around a dozen. The update frequency does not seem very high, so it would be good to confirm the current maintenance status before adopting it.
- Running it requires a Java runtime.
- The checksum is MD5. For the purpose of error detection at accessioning time, the practical problem seems small, but if you are newly designing an identity anchor, there are cases where you would choose SHA-256.
While it has high value as a reference implementation of concepts and formats, whether to build it into a production workflow as-is over the long term is something to judge according to each institution's circumstances.
Division of Labor with My Own SIP-Creation Tool
Separately, in the context of working on a lightweight SIP-creation tool, I considered whether "I should have DataAccessioner-equivalent functionality myself." I share my current thinking.
First, it seems I do not need to generate the METS structMap myself. That Archivematica uses structMap to retain the pre- and post-arrangement structure is a matter of that implementation's convenience; the information needed for the before↔after correspondence can be aggregated into each file's "original path + identity (hash)." Generation of structMap and PREMIS can be left to the downstream ingest, and having the accessioning side output a full METS tends to be excessive. However, this judgment presupposes that the downstream (such as Archivematica) takes on the before↔after reconstruction via originalName and the like. When the downstream has no preservation system or is heterogeneous, the accessioning side needs to retain more original-state information.
On the other hand, what is easy to overlook is that many SIP-creation tools "just package a single point-in-time folder they are handed," and have no distinction between before-arrangement and after-arrangement. If you finish arrangement first in Finder or the like and then hand it to the tool, the tool only sees the post-arrangement state, so the pre-arrangement structure is not recorded.
What DataAccessioner shows is the order that fills this gap.
- Run it once before arrangement, and fix each file's original path + checksum (+ optionally technical metadata) as an accessioning record.
- Then arrange and turn it into a SIP.
- Match the two by hash to trace before↔after.
Applying this to my own tool, by adding an accessioning snapshot that bundles the original path to the checksum.sha256 (identity) and DFXML (Digital Forensics XML, path information) I already output, I should be able to express the core of DataAccessioner. Even without reimplementing full FITS or METS, I can take on the role of "fixing the pre-arrangement state that is hard to recover downstream." In my own tool, I adopt this policy (leave an accessioning-time snapshot and delegate structMap generation to the downstream).
In Lieu of a Conclusion
DataAccessioner is a desktop tool for recording accessioning by capturing technical metadata via MD5 and FITS before arrangement or SIP creation. It divides in role from Archivematica by the boundary of custody (Archivematica also performs accessioning and characterization at the Transfer stage, and handles arrangement and SIP creation inside, so there is functional overlap), and DataAccessioner takes on the system-independent accessioning record before entering the preservation system. Within what is publicly available, its update frequency is not high and the checksum is MD5-based, so it is easier to treat it as a reference implementation of concepts and formats. Even when incorporating it into a homegrown tool, my current assessment is that the core can be reproduced with an "original path + SHA-256 accessioning snapshot" rather than a full METS / FITS.
Comments
…