🗂️Characterizing a Set of Files by Running Brunnhilde on macOS (Format Identification with Siegfried)

This article was co-authored with generative AI. Facts have been checked against public documentation where feasible, but errors may remain. Please verify primary sources before relying on this for important decisions.

When transferring born-digital records (records created digitally from the start) or digitized files to an archive, there is a step that mechanically inventories "what formats are contained, how much of each, and with what risks." Brunnhilde is a command-line tool for doing this characterization all at once.

In this article, I install Brunnhilde on macOS (Apple Silicon), run it against a sample set of files, and confirm it through to producing a format-identification report. Virus scanning (ClamAV integration) was covered in a separate article, so here I focus on format identification and proceed with ClamAV integration disabled (-n).

Relationship Between the Tools

Brunnhilde is not a standalone identification engine but a wrapper that calls several tools and aggregates them into a single report. At its core is Siegfried (format identification), with ClamAV (virus scanning) and bulk_extractor (scanning for PII = Personally Identifiable Information) as optional callable components.

Each role is as follows.

• Siegfried: Matches the file contents (byte sequence) against PRONOM signatures (the format registry of the UK National Archives) and identifies the format by PUID (PRONOM Persistent Unique Identifier, e.g., fmt/18). It is Brunnhilde's required dependency.
• ClamAV: Virus scanning. Optional; can be disabled with -n/--noclam, and is auto-skipped on Windows.
• bulk_extractor: Pattern extraction of PII and the like. Runs only when -b is given.
• tree, etc.: An auxiliary process that outputs the directory structure to tree.txt. When a disk image is the input, tsk_recover (SleuthKit), fiwalk, unhfs (HFSExplorer), and others are also called for extraction.
• For checksums, rather than Brunnhilde using an external tool like md5deep, it uses Siegfried's hash output (sf -hash, md5 by default). Duplicate detection is also done based on that hash column.

The Layered Relationship with BitCurator and SIP Creator

These tools are sometimes described in a single line like "BitCurator → Brunnhilde → Siegfried → …," but the "contains" relationship that each stage refers to differs from stage to stage. Specifically, three kinds of relationship are mixed: the environment "bundling" a tool, a wrapper or app "calling" a tool, and a tool "referencing" data.

Organized, it is as follows.

• BitCurator is an Ubuntu-based environment (distribution) for archivists, which "bundles" Brunnhilde, Siegfried, bulk_extractor, and others. Note that it directly bundles Siegfried on its own, not only via Brunnhilde.
• Brunnhilde is a wrapper that at runtime "calls" Siegfried and (if enabled) ClamAV as child processes. Siegfried is a required dependency.
• Below Siegfried is not a tool but "data": the PRONOM signature database (plus, optionally, Wikidata / freedesktop MIME).
• CCA-Public/sipcreator (SIP Creator) is a GUI app that "calls" Brunnhilde. It is a separate codebase from BitCurator, and rather than being a part contained within BitCurator, it is in a side-by-side relationship as a user of Brunnhilde (though note that it is described as "intended for use with BitCurator 4," and is often run inside a BitCurator environment).

Because each layer runs on its own, you can run Brunnhilde → Siegfried on macOS without using BitCurator, as here, and you can also run sf directly.

Setup

I install Siegfried via Homebrew and Brunnhilde as a Python package (pip). I also install tree, used to output the directory structure.

# Siegfried (the format-identification engine, a required dependency)
brew install richardlehane/digipres/siegfried
sf -version

# Used for directory tree output (without it, tree.txt is just empty but the run still succeeds)
brew install tree

# Brunnhilde itself (put it in a venv so as not to pollute the system Python)
python3 -m venv .venv-brunnhilde
.venv-brunnhilde/bin/pip install brunnhilde

The Siegfried installed via Homebrew bundles a signature database; this time it was siegfried 1.11.4 / signature default.sig (as of 2026-06-06). The Brunnhilde version is 1.9.6.

After installation, the entry point is named brunnhilde.py (not brunnhilde).

Preparing Sample Data

I use a script that generates a sample simulating transfer records. In addition to a variety of formats — PDF / PNG / DOCX / XLSX / CSV / TXT — it builds a tree that deliberately mixes in duplicate files and files prone to being unidentified (.dat), so that Brunnhilde's demo shows well.

sample-data/2026-移管-対象部署
├── その他/unknown_export.dat
├── 移管メモ.txt
├── 議事録/_バックアップ_第1回議事録.pdf
├── 議事録/2024年度第1回_議題メモ.docx
├── 議事録/2024年度第1回運営委員会_議事録.pdf
├── 議事録/2024年度第2回運営委員会_議事録.pdf
├── 写真/式典_2024.png
├── 写真/式典_2024_集合写真.png
├── 名簿/教職員名簿_2024.csv
└── 名簿/教職員名簿_2024.xlsx

議事録/2024年度第1回運営委員会_議事録.pdf and 議事録/_バックアップ_第1回議事録.pdf have identical contents, serving as the duplicate-detection demo.

Running It

Turning off ClamAV integration (-n), I specify the input directory and the output destination.

.venv-brunnhilde/bin/brunnhilde.py -n sample-data out/brunnhilde-test

In the output destination, the human-readable report.html, Siegfried's raw result siegfried.csv, the aggregated csv_reports/ (formats.csv / duplicates.csv / warnings.csv, etc.), and tree.txt are lined up.

Looking at the Report

Opening report.html in a browser, the per-format aggregation, duplicate files, unidentified files, warnings, and so on are gathered onto a single page.

The results for this sample were as follows.

Metric	Value
Total files	10
Distinct files	9 (including 1 duplicate pair)
Identified formats	7
Unidentified files	0

The per-format aggregation is as follows. PDF is 3, PNG is 2, and the others are 1 each — classified as expected.

Format	PUID	Count
Acrobat PDF 1.4	fmt/18	3
Portable Network Graphics	fmt/11	2
Plain Text File	x-fmt/111	1
Microsoft Word for Windows	fmt/412	1
Microsoft Excel for Windows	fmt/214	1
Data File	fmt/1730	1
Comma Separated Values	x-fmt/18	1

The two PDFs with identical contents were listed as duplicates under the same checksum (dd3eecbb…).

"Unidentified 0" but Two "Identified by Extension Only"

On the report it says "Unidentified files 0," but looking at warnings.csv, two files, .dat and .csv, carried the warning match on extension only.

File,...,Basis for ID,Warning
.../その他/unknown_export.dat,...,extension match dat,match on extension only
.../名簿/教職員名簿_2024.csv,...,extension match csv,match on extension only

This means that the PUID could not be identified by byte-sequence signature and was assigned on the basis of the extension alone. Because Siegfried assigns fmt/1730 (Data File) or x-fmt/18 (CSV) from the extension, they count as "identified" in the aggregation, but their confidence is treated as lower than a signature match. fmt/1730 (Data File) is itself defined on the PRONOM side as a catch-all "generic term for accepting .dat-type files that do not yet have a signature, by extension alone," a typical case of extension-only matching. It happens readily with plain-text types (such as CSV) and .dat files whose contents are undetermined, so when reading characterization results, checking this warning as well gives a picture closer to reality.

Supplementary Notes

• Even if tree: command not found appears at runtime, only tree.txt becomes empty and the identification processing itself completes. Install brew install tree only if you want to keep the directory tree.
• If you want to check with ClamAV integration included, that is covered in a separate article (Verifying ClamAV on macOS). In an environment with a resident antivirus, note that test files may be quarantined by on-access scanning.
• The Brunnhilde venv installed for verification can be removed with rm -rf .venv-brunnhilde if no longer needed.