You are currently browsing the monthly archive for October 2008.

Well, the short answer is 2037, plus or minus a few years.

That’s not a flippant answer, either. All forms of digital preservation really will stop around 2037, unless some kind of energy supply breakthrough happens.

How do we arrive at this date?

Let’s go thorough this step by step.

Technological collapse

The fundamental premise behind what follows is that digital preservation cannot survive the collapse of our technological civilisation. If you disagree with that premise, that’s fine, but you might as well stop reading now. It is always good to be clear about our premises before we begin. If you agree with the premise, then let’s carry on.

Paper records, if stored or hidden in a substantial box, can last centuries without any active preservation measures being undertaken. The civilisation which created those paper records might collapse, but the box could survive. A future civilisation can then discover the box, realise there is a message-bearing medium inside, and work out what it says. (It’s even better if the records are on stone. There’s a three thousand year gap between the Assyrian messages at Behistun and the 19th century European explorers who mapped and transcribed them, but that gap did not stop linguists from deciphering the cuneiform messages.)

Digital media have much shorter timespans. It is doubtful that a hard drive will be able to spin and deliver its data a few centuries after our society has collapsed. Technologically-dependent data storage therefore cannot survive massive societal collapse in the same way that non-dependent data can. 

We have arrived at the first possible answer to our question, which we will call answer A1:

A1: Digital preservation will come to an end when technological civilisation comes to an end.

Now let’s start to pin this down. When will technological civilisation come to an end?

Modern computing is wholly dependent upon hardware which in turn is wholly dependent upon fossil fuels for its creation and maintenance. The servers or CDs which preserve our data incorporate plastics which have been refined from crude oil supplied by OPEC. The dust-free clean rooms in which the chips are made are kept clean by energy derived from burning hydrocarbons. The finished computers are distributed globally by diesel-burning ships, which deliver them to ports from which the machines are then placed onto diesel-fuelled trucks for final distribution to warehouses and shops. The world’s ICT infrastructure is maintained by people who get to and from work in vehicles powered by petrol. Without crude oil none of this would happen.

The oil basis of modern ICT is an issue which gets raised from time to time. In December 2007 New Scientist reported that “computer servers are at least as great a threat to the climate as SUVs or the global aviation industry,” due to their carbon footprint. A 2004 United Nations study showed that the construction of an average 24-kilogram computer and 27-centimetre monitor requires at least 240 kilograms of fossil fuel, 22 kilograms of chemicals and 1,500 kilograms of water – or 1.8 tons in total, the equivalent of a sports utility vehicle.

Take away all this oil, plastic, petrol and diesel, and the world’s ICT structure becomes unsustainable. Motherboards become trickier to manufacture if you only have wood and brass. Gathering together the components and then distributing the finished machines becomes harder if you are dependent on sailing ships, horse-drawn carriages and barges for transport.

We can now refine our earlier answer. If we agree that digital preservation will come to an end when modern technological civilisation comes to an end, and if we then agree that modern technology is currently wholly dependent on oil and oil-derived plastics for its maintenance, then we arrive at the following statement:

A2: Digital preservation will end when the oil supply comes to an end.

But when will the oil supply come to an end? Never, in a sense, because at some point it will become too uneconomical for the world to drill out the last remaining drops. There will always be some oil left in the earth. Sadly that’s no help to us, because we will be back in the stone age by then.

A better question is, when will the oil supply start to run out?  – because that’s the point at which civilisation crashes; that’s the point at which any particular country can only increase its own oil and plastic by taking away oil and plastic from another country. And from that date, year on year, there will be less oil and plastic than the year before.

I’m no geologist, so let’s go to the experts on this one. The EIA (Energy Information Administration) is the energy data arm of the US government. In 2004 the EIA published a report on Long Term Oil Prospects, which looked at exactly this question. The report’s authors considered a number of likely scenarios for both (a) the total amount of oil in the ground and (b) the increase in demand for oil as time progresses. Then they mapped out all these scenarios.

This graph shows the three main scenarios, with the central one being the likeliest, as it is based on a world total oil production figure of about 3 trillion barrels of oil, which is the US Geological Survey’s assessment. The overall curve has a sharkfin shape. World oil supply rises upwards with a 2% annual growth rate until it peaks and then suddenly falls, when the world’s oil wells cannot meet demand. The peak comes in 2037.

So, for our purposes we can say that, as digital preservation will come to an end when the oil supply comes to an end, and as the oil supply will come to an end in 2037, we can then say that:

A3: Digital preservation will come to an end in 2037.

Certainly we will have bigger problems in 2037 than simply digital preservation. One problem which springs to my mind, as a UK citizen, is starvation. Much of our food in the UK is grown overseas (using fossil fuel-based fertiliser) and then shipped across. U-boat warfare almost starved Britain in the 1940s, yet back then we had more land under arable cultivation, a smaller population to feed, and a bigger proportion of our population was involved in agriculture. When the world’s shipping stops in 2037 digital preservation will be less of a priority than personal survival. As Peter Goodchild recently wrote, when the oil supply stops “our descendants will be smashing computers to get pieces of metal they can use as arrowheads.”

That doesn’t sound very optimistic.

No, but the oil might not run out in 2037. It might run out later (although the crash will be bigger).

Some people, such as the Peak Oil crowd who hang around at The Oil Drum, think the oil supply might be running out just about now, but I’m no energy expert so I’ll stick with the US government’s EIA on this one.

The longer we have, the better are our prospects at longer term digital archiving, because it gives the world more time to create and roll out a new ICT structure, one which doesn’t use oil-derived plastics, or depend on oil for distribution, power and maintenance. On the other hand, the shorter we have, the worse our prospects will be.

Local authority archive services in the UK tend to have very good collections of parish records and local government records, but have poorer collections of business archives. In my experience business records are only deposited when the business itself has been liquidated, or been taken over, or when it has simply vanished from a building and another company has moved into the premises to discover a heap of old paper and ledgers. I have taken part in at least one instance of “rescue” archiving of this nature, when our team waded through mud inside a soon-to-be-demolished warehouse, picking 19th century volumes out of the dirt.

This kind of rescue activity only happens because it’s easier for the new company, or for the official receivers, to persuade an archives service to sort out all this old paper than it would be for them to sort it out themselves.

But the reverse is true for digital archives. When a modern business goes bust, many of its records will exist only in electronic form. The receivers’ primary job is to identify and sell the assets of the business and dispose of the remainder. For most businesses, the only electronic record which would have value as a saleable asset would be the list of its customers’ contact details. Privacy policies now explicitly state that customers’ information may be sold in the event of bankruptcy. Here’s one I found at random:

Business sale
If [Name of company] Ltd or any of its lines of business is sold, pledged or disposed of as a going concern, whether by merger, sale of assets, bankruptcy or otherwise, then the user database of [the company] could be sold as part of that transaction and all User Information accessed by such successor or purchaser.”

Customers’ details, therefore, are not going to be handed over free to the nearby archives office. They will instead be sold to the highest commercial bidder. What about all the other electronic files – the personnel records, publicity photos, advertising material, accounts?

I suspect that most official receivers would treat the ICT hardware as either another asset for resale (if the kit was recent or high spec), or they would dispose of it to professional ICT equipment salvagers, who would ship it over to the developing world. Either way the data on the computers would get wiped. It is unlikely that they would contact their local archivists to suggest we come over with a truck to remove all the old servers and PCs. It also seems unlikely that they would wait for us to turn up with portable hard drives, power up all the old equipment, and work out what data we want to transfer across from dozens of separate servers and desktop machines.

The inheriting organisation will always be under pressure to take the easiest and cheapest way to dispose of a predecessor’s assets, which in practice probably means that data will be wiped and the hardware sold on. We are therefore looking at potentially a very large loss of historical business data.

Bankruptcy has been recognised in the past as a threat to digital preservation: read this from “Requirements for Digital Preservation Systems: A Bottom-Up Approach” in D-Lib magazine, back in November 2005:

Organizational Failure. The system view of digital preservation must include not merely the technology but the organization in which it is embedded. These organizations may die out, perhaps through bankruptcy, or their missions may change. This may deprive the digital preservation technology of the support it needs to survive. System planning must envisage the possibility of the asset represented by the preserved content being transferred to a successor organization, or otherwise being properly disposed of. For each of these types of failure, it is necessary to trade off the cost of defense against the level of system degradation under the threat that is regarded as acceptable for that cost.

Paper records can easily survive organizational failure, but electronic records will have a much tougher time.

These thoughts have been prompted by the slowdown in Western economies and the possibility of a forthcoming global recession. Some businesses have already gone bust, and the omens are not good for many others. For 99% of these companies, their electronic records will evaporate with the death of the business.

It’s possible that I’m worrying unnecessarily here. I would love to hear if anyone knows of instances where local archive repositories have been offered electronic data from former businesses which have gone bankrupt, rather than from live, viable companies.