This article was co-authored with generative AI. Proper nouns, figures, and technical specifications in the article have been checked against public information where feasible, but errors may remain. For accurate information, please consult the primary sources at the end. This article focuses on the technical aspects of data formats and architecture.

This is the fourth (and final) installment in a series introducing representative large-scale digital-humanities projects. Related articles: Venice Time Machine / Sailing Letters / Stanford ORBIS

First, What Is a "Gazetteer"

A Gazetteer is what in Japanese is called a "ๅœฐๅ่พžๅ…ธ" (place-name dictionary) or "ๅœฐๅ็ดขๅผ•" (place-name index). Look up a place name, and you learn that place's coordinates, alternate names, and place type (city, port, province, etc.) โ€” in short, an index database of places.

World Historical Gazetteer (WHG) is a project that tries to build this "at the scale of world history, and including temporal information," and moreover to make it a platform that interconnects historical datasets from around the world via place names. It is operated by the World History Center at the University of Pittsburgh in the U.S., and the historian Ruth Mostern leads the project. Development began in 2017 with a grant from the National Endowment for the Humanities (NEH), and after V1 (2017โ€“) and V2 (2021โ€“), Version 3 (from 2024) is now in operation.

In this article, I focus on the technical challenge that WHG tries to solve and the data format at its core.

The Technical Challenge It Tries to Solve: Reconciling Place Names

What WHG tackles, at bottom, is the technical challenge of "place-name reconciliation."

Place names change over time. For example, Edo โ†’ Tokyo (1868), Constantinople โ†’ Istanbul (officially renamed 1930), Siam โ†’ Thailand (1939), Ceylon โ†’ Sri Lanka (1972). A human knows they are the same place, but a computer recognizes them only as separate strings.

Furthermore, historical databases built by researchers around the world record the same place under scattered names and IDs, such as Edo / ๆฑŸๆˆธ / Yedo. This makes it impossible to join and analyze multiple databases.

WHG solves this problem by "linking each dataset's place names to a common authority."

Mechanism: Reconciliation to a Common Authority

When a user uploads their dataset's place names to WHG, WHG reconciles them against existing place-name authorities. The core reconciliation targets are two globally used authorities.

  • Wikidata (a structured knowledge base)
  • GeoNames (a place-name database)

In WHG's Version 3, this reconciliation index is built at a scale of about 10 million GeoNames records + about 3.6 million Wikidata records = about 13.6 million total. When reconciliation succeeds,

  • that place name's alternate names (Tokyo / Edo / Yedo โ€ฆ) are automatically linked
  • coordinates such as latitude and longitude are filled in
  • it is connected to other datasets that reference the same place

In this way, historical databases that had been isolated become mutually connected via a common "place." As of Version 3 in 2024, WHG links more than 70 public datasets/collections, and that number keeps growing.

Core Technology: Linked Places Format (LPF)

The biggest reason WHG is regarded as important in the technical community is that it led the development of a data format called Linked Places Format (LPF).

In the world of map data, a standard format called GeoJSON is widely used. But while GeoJSON is good at describing "the coordinates of a point," it was poor at handling temporal information โ€” "a place at which point in time."

LPF was designed to make up for this weakness. Technically,

  • it is valid GeoJSON and at the same time an implementation of GeoJSON-T (an extension spec that adds temporal attributes to GeoJSON)
  • it can express time-stamped geographic information such as "this place name was used from year X to year Y" and "this place belonged to which territory during which period"
  • it can also hold, in structured form, alternate names (toponyms), external links to authorities (IDs of Wikidata/GeoNames, etc.), place types, and more

Additionally, a simplified version, LP-TSV (Linked Places Delimited), that lets you easily create place-name records in tabular form, is provided. It is a practical consideration: you can prepare and upload place-name data as if using a spreadsheet, without writing complex JSON.

"If everyone creates data in the same format, the data automatically connects to each other" โ€” this standardization is WHG's essential contribution.

Architecture

WHG's backend is implemented with the following composition.

  • PostgreSQL (a relational database): the canonical store for place-name records
  • Elasticsearch (a search engine): the index for fast place-name search and reconciliation
  • Django (a Python web framework): binds the above two and manages the upload, reconciliation, and publication workflow

This is the "canonical data in an RDB, search in an Elasticsearch index" composition, commonly adopted in search-heavy systems.

Why This Matters

WHG's significance lies less in flashily "showing" something on its own, and more in becoming a hub (a nexus) that connects other projects.

  • Interoperability: separately built historical data can be connected via common places
  • Time-stamped place-name management: it can accurately handle "when, and where" (essential to historical research)
  • Open standards and authorities: connects to the open format LPF and the global authorities Wikidata/GeoNames

The other projects introduced in this series โ€” Venice Time Machine, Sailing Letters, ORBIS โ€” also cannot ultimately interconnect unless they align "which place they are talking about" by a common standard. WHG seeks to serve as that common infrastructure.

Limitations and Caveats

  • Limits of coverage: recording all place names in world history is fundamentally impossible, and coverage depends on the linked datasets
  • Difficulty of reconciliation: judging "whether this place name and that one are truly the same" is not self-evident and requires expertise
  • Variability of quality: precision and granularity differ from one linked dataset to another

Summary

World Historical Gazetteer is not a flashy visualization project. But it takes on work whose importance grows year by year as a foundation of the digital humanities: "connecting historical data from around the world through the common language of time-stamped place names."

Its technical core lies in the open data format LPF (an implementation of GeoJSON-T) and reconciliation to Wikidata/GeoNames. Before "connecting" data, first align "places" by a common standard โ€” WHG is the unsung workhorse that builds this address book of world history.

Reference Links