This article was co-authored with generative AI. The proper nouns, figures, and history in the article have been cross-checked against public information where feasible, but the historical-background descriptions in particular may contain errors. For accurate information, please check the primary sources at the end. This article focuses on introducing the technical mechanisms.
This is the first article in a series introducing representative large-scale projects in the digital humanities (humanities × digital technology). Related articles: Sailing Letters / Stanford ORBIS / World Historical Gazetteer
What this project is
The Venice Time Machine (VTM) was a project that tried to mechanically digitize and structure the vast public records left by the Republic of Venice over a long period, building a "database that lets you search and reconstruct the past like a map."
Switzerland's EPFL (École polytechnique fédérale de Lausanne) and Venice's Ca' Foscari University jointly launched it in 2012. The central figure is Frédéric Kaplan, an information engineer who leads EPFL's Digital Humanities Laboratory. Kaplan expressed this vision as the "Big Data of the Past" (the title of a 2017 paper by Kaplan and colleagues), or, in TED talks and elsewhere, as "Google Maps of the past."
In this article, we look mainly at this project's technical mechanisms and at the data-quality and provenance-management challenges that were exposed there.
The technical pipeline: turning mountains of paper into data by machine
VTM's processing was broadly conceived in the following four stages. This is the technical heart of the project.
| Stage | Content | Technical point |
|---|---|---|
| 1. Scanning | Photograph historical documents into digital images | About 1,000 pages per hour with a semi-automatic scanner |
| 2. Deciphering the text | A machine converts images of handwritten characters into text | Handwritten text recognition (HTR) — far harder than printed-type OCR |
| 3. Pattern extraction | Mechanically pick out recurring elements such as personal names, place names, and phrases | Named-entity extraction via natural language processing |
| 4. Graph building | Using the extracted elements as clues, connect documents to one another | Reconstruct the relationships of people, places, and time as a "graph (mesh)" |
The difficulty of handwritten text recognition (HTR)
The single hardest part technically is stage 2, HTR (Handwritten Text Recognition).
OCR that reads printed type is already at a practical level, but handwritten documents from hundreds of years ago combine factors such as:
- Character forms and idiosyncrasies vary by writer
- Abbreviations, ligatures, and spellings specific to the time
- Ink bleeding, paper deterioration, and marginal notes
which make them an extremely difficult target for a machine. VTM was a technically challenging project in that it tackled this HTR at scale.
Graph building = reconstructing social networks
Stage 4 is also important. For example, it extracts from documents a fact like "a certain person sold a certain piece of land to someone in a certain year," and expresses the connections of people, places, and time as a graph. Accumulate a great many of these, and you can reconstruct the social network itself — the past city's transactional relationships, kinship, and ownership relations — as data. This was VTM's methodological aim.
In fact, a model reconstructing the transformation of the city of Venice was exhibited at the 2018 Venice Biennale (the architecture exhibition) and elsewhere.
The scale of the target data
The Venice State Archive that VTM targeted holds administrative documents spanning over 1,000 years, about 80 km in shelf length. Under a thoroughgoing culture of record-keeping, documents on taxes, registration, land, trials, and so on have accumulated in vast quantities.
This is an enormous research resource, but it is also "an amount no human can read through," which was the motivation for bringing in machine processing.
The challenge that surfaced in 2019: data quality and provenance management
VTM is also known as a case that illustrates the "pitfalls of large-scale digitization projects" in the digital humanities.
In 2019, the scientific journal Nature reported that VTM had been suspended (Nature 2019). According to the report, the Venice State Archive side reconsidered its collaboration with EPFL and pointed out serious problems with the roughly 8 terabytes of data collected up to that point. The point of contention lay not so much in the technology as in how the data was handled.
- The collected data was described as "essentially useless"
- Because the digitization work procedures had not been sufficiently recorded, it was impossible to verify after the fact "which original was photographed and processed with which settings"
- It did not follow the international frameworks for digitally preserving public records (such as InterPARES, the practices that guarantee the authenticity and provenance of records)
This is an extremely important technical lesson. No matter how much and how fast you scan, unless you record and guarantee "how that data was created (provenance)," it cannot be trusted as research material. It threw into relief a universal issue in building data infrastructure: which to prioritize, "speed and scale" or "accuracy and verifiability."
What came afterward
VTM itself was suspended, but its vision was carried over into a larger framework of "digitizing the past of European cities by similar methods" (Time Machine Organisation / Time Machine Europe). Note that in the EU's mega-scale research grant "FET Flagship," the Time Machine vision advanced to the stage of being proposed and drawing attention as a candidate.
Note (funding sources): VTM is often introduced as having "received a large ERC (European Research Council) grant," but as far as I can confirm, the main funding sources were the Lombard Odier Foundation (from 2014), the Swiss National Science Foundation (SNSF), and the EU's Horizon 2020, and I could not confirm an ERC grant.
Lessons for engineers and data practitioners
VTM is rich in implications for everyone who builds data infrastructure.
- Possibility: The direction of turning large amounts of handwritten historical material into data with HTR and natural language processing, and visualizing relationships as a graph, is still valid and continuing to develop
- Lesson: When designing a pipeline, always record the metadata of each processing stage (when, what, and with which settings it was processed). Provenance records cannot be added after the fact
- Collaboration: A structure in which engineers and specialists in historical material and records (archivists) proceed while respecting each other's practices is indispensable
The goal of "turning the whole of the past into data" cannot be completed by technology alone. Only with a design that guarantees the provenance of the data, and collaboration with specialists, does the collected data become something "usable."
Comments
…