Record Linkage: project workshop and work in progress

We’re holding an afternoon workshop on record/data linkage in Sheffield on 4 November. The aim is to explore the challenges and rewards of applying automated nominal record linkage to large-scale historical datasets, with all their variability, fuzziness and uncertainties, but we’d also very much welcome participants and insights from all fields concerned with data linkage including social sciences, health sciences and computer science. In addition to presentations about our work in progress on 90,000 19th-century prisoners and convicts, we have guest speakers who will bring extensive experience of historical record linkage projects to the discussion. It’s free to attend and anyone with an interest, at any stage of their academic career, is welcome (I’d particularly love to see plenty of PhD students!). More info can be found on our website here (and there’s also a programme to download).

Record linkage is really at the heart of the Digital Panopticon project’s goals to explore the impact of the different types of punishments on Old Bailey Online defendants between about 1780 and 1875 (along with working on data visualisations for exploring, presenting and communicating the data and research findings). Our research questions include: How can we improve current record-linkage processes to maximise both the number of individuals linked across different datasets and the amount of information obtained about each individual? What is the minimum amount of contextual information needed in order to conduct successful large-scale record linkage of data pertaining to specific individuals?

I’ve blogged in the past about problems associated with historical record linkage where you don’t have handy unique IDs (like, say, National Insurance numbers): names are often crucial but are highly problematic, and problems with a source like Old Bailey Online that tells us about sentences but not actual punishments. Those are among our biggest headaches with Digital Panopticon.

There are a lot of missing people when we link OBO to transportation records, and a lot of possible reasons for linking to fail. There might be errors in the data created at almost any point between the making of the original source and our production of a specific dataset to feed to the computer: eg, if you’re extracting a London-only subset from a national dataset and you’re not careful, you might also end up with records from Londonderry. Oops. (“You” there is an euphemism for “I”. )

Then there are problems caused by spelling variations in names, or the use of aliases and different names. And the problem of common names. As I blogged before: “How do you decide whether one Robert Scott is the same person as another Robert Scott, or someone else altogether?” But that gets much worse when the name in question is “Mary Smith”.

And the fails that are due to the gaps in our data: Were they pardoned? Did they die in prison or on the hulks before they could be transported? And so we are on a quest to track down sources that can tell us these things and fill the gaps (not all of which have been digitised; some of which have probably not even survived, especially from the 18th century).

Irreconcilable conflicts can emerge between different sources (eg, different trial dates and places). At this point we have to turn to the specialist knowledge of the project team on how, when and where particular sources were created so we can attempt to rate the relative reliability of two conflicting sources. But how are we going to handle those weightings when we’re dealing with  thousands of people and the links are all probables anyway? (Just because source A is generally more reliable for a certain piece of information than source B doesn’t mean A is always right and B is always wrong if they’re in conflict.)

So there will be plenty to discuss at the workshop and for the next three years!

For tasters of what we’ve been getting up to so far:

CFP: Sensing the Sacred: Religion and the Senses, 1300 – 1800

Interdisciplinary conference, University of York, 21-22 June 2013

The burgeoning field of sensory history offers a fertile ground for reconsideration of religious studies across disciplinary boundaries. We welcome papers from anthropologists, archaeologists, art historians, historians, literary scholars, musicologists, philosophers, theologians, and any other interested parties. …

Proposals (max. 300 words) for papers of 20 minutes are welcomed both from established scholars, and from postgraduate students. Applications from panels of three speakers are encouraged, as well as individual proposals.

Digital Humanities Congress 2012

University of Sheffield, 6 – 8 September 2012

Registration is open now!

The University of Sheffield’s Humanities Research Institute with the support of the Network of Expert Centres and Centernet is delighted to invite you to a new conference intended to promote the sharing of knowledge, ideas and techniques within the digital humanities.

At Sheffield we understand the digital humanities to mean the use of technology within arts, heritage and humanities research as both a method of inquiry and a means of dissemination. We’re therefore excited to have a varied programme with speakers from disciplines across the arts, humanities and heritage domains.

Our keynote speakers are:
Professor Andrew Prescott (Head of Department, Department of Digital Humanities, King’s College London)
Professor Lorna Hughes (University of Wales Chair in Digital Collections at the National Library of Wales)
Professor Philip Ethington (Professor of History and Political Science, University of Southern California and Co-Director of the USC Center for Transformative Scholarship)

Full programme (pdf)

A few calls

1. CFP: Reading Conference in Early Modern Studies 2009

The next annual meeting of the Reading conference on early modern studies will be held on 6-8 July 2009, with an informal theme of ‘Authority and Authorities’. “The Reading conferences are as broadly based as possible, reflecting the most interesting developments in current research. Accordingly we welcome proposals for either complete sessions or individual papers from scholars in any discipline or any area of early modern studies, including Atlantic, European and imperial perspectives…” (Full details at the link.)

2. Guest bloggers wanted

Brandon Watson is looking for guest bloggers at his early modern history of philosophy blog Houyhnhnm Land; not necessarily history of philosophy specialists – “the posts have to be on some facet of early modern thought (or approaches thereto), but just about anything falling under that label would work. I’d love, for instance, to get historians of all kinds, literary scholars, and the like adding their two cents; I’d also love specialists from outside the early modern period looking at how later periods viewed the early modern period or how earlier periods prepared for it; and so forth.”

3. Carnivalesque and History Carnival hosts needed

I really, urgently need History Carnival hosts for November and December (1st of the month). Please email me as soon as possible if you could do one of these:

A host is also needed for the November ancient/medieval edition Carnivalesque (same email address will do, or

Both carnivals will also need hosts next year, so if you’re too busy in the immediate future but might like to take one on later, get in touch.

Old Bailey Online: now from 1674 to 1913 (check it out before it collapses)

Well, I was a little cryptic the other week, but tomorrow it all goes public (and we kind of expect it to crash at some point – I’ll be almost disappointed if it doesn’t…),* and today there is a pretty nice feature in the Observer.

[Monday update… creak… groan… thud… Sorry, folks. It should get back to normal in a day or two…]

So here it is: the Proceedings of the Old Bailey Online 1674-1834 is now the Proceedings of the Old Bailey and Central Criminal Court 1674-1913.

This doesn’t only mean that you can now search for 200,000 trials held at the Old Bailey over a period of 2 and a half centuries. The other new set of goodies is the full text of (almost) every Ordinary of Newgate’s Account between 1690 and 1772 (in the next few months this should expand to a full archive of every known surviving Account from c.1674 onwards).

I’ve written here before about these grimly fascinating pamphlets. They’ve been used by a number of historians, including Andrea Mackenzie and Peter Linebaugh, but the surviving pamphlets have been scattered across a number of different libraries and archives. From now on they’ll be together in one fully searchable digital archive. Plus, I’m in the process of completing a database that links every convict mentioned in the Accounts to their trial, providing it has a surviving report (perhaps 3/4 of the links have already been made).

This should make for some interesting research possibilities. For example, historians often argue that women who successfully ‘pleaded their bellies’, ie had their death sentence postponed on grounds of being pregnant, usually escaped hanging. In fact, we say that in our own background section. But I’m not so sure. Through the process of cross-referencing trials and Ordinary’s Accounts, I’ve already discovered several women whose sentences were respited for pregnancy but subsequently carried out (eg in September 1695. So what I’ll be asking (once I’ve finished making the damned links) is: how many were executed and how many were permanently reprieved? Have we historians been getting it wrong? Answering those questions wasn’t impossible before now, but it would have been extremely difficult. And there will, no doubt, be many more possibilities like this.


The other news, because I haven’t been plugging it enough and you’ve probably all forgotten, is that we’re holding a conference in July to celebrate the relaunch: The Metropolis on Trial, in the throbbing metropolis of… Milton Keynes. If you’d like to attend, registration is open and you can download a booking form at the website. If you want to book the accommodation we’ve arranged at discount rates, you need to send the form in by the end of May at the latest and preferably as soon as possible. There is a 2 person room sharing option which is really good value (if you’re skint and looking for someone to share with maybe we can put people in touch here – leave a note in comments).



(Note that old links will continue to work for a few months, and we may well set up proper redirection at some point.)

Old stuff on OBP at this blog: Old Bailey category and the Old Bailey Symposium.

Old Bailey Files at The Head Heeb.

*Already this morning some searches have been very slow, which is not a good sign.

Wars, Conferences and Blogs

For those interested in the British Civil Wars, a symposium is being held next July in Hull.

In a lecture delivered to the Royal Historical Society in December 1983, John Morrill concluded with the observation that ‘The English civil war was not the first European revolution: it was the last of the wars of religion’. … This symposium aims to recognise the importance of Morrill’s interpretation, and to move it forward with reference to scholarship on political and religious thought that has emerged since 1983. While it will be partly concerned with the period of the 1640s, it also aims to draw out elements of the links and tensions between politics and religion that define the long seventeenth century. Central to the symposium will be a critical engagement with Morrill’s original argument: in what ways is it still persuasive, and in what areas might it be revised?

But what really struck me was that the organisers are using a blog as a website for the symposium. A smart idea: it’s free and not dependent on a university department’s web space, so interesting material can be left up afterwards for as long as you want; it’s simple to set up and can be used to post news and information about the event quickly and easily (with RSS feeds, of course), as well as paper abstracts and even copies of the papers themselves for pre-circulation (though that’s not something we do that much in history usually…). And then, think about the possibilities for discussions with people who can’t actually attend the event. And podcasts! And…

It’s a really obvious thing to do with a blog, when you think about it, isn’t it?

Update: And so, of course… I have to have one too, don’t I?

CFP: The Metropolis on Trial

Really, it would be shockingly negligent of me not to plug our project conference here, now wouldn’t it? For all those of you interested in The Old Bailey Online, crime, justice and so on between the 18th and 20th centuries (not just in London – we’re looking out for comparative papers), the project conference will be next July, following the launch of the Proceedings of the Central Criminal Court in 2008.

In addition to the general descriptions in the CFP below, there are a couple of planned panels that might particularly interest you folks.

First, we’d like to have a panel specifically on teaching with the OBP. My friend Chris Williams at OU is in charge of this one: “I’m interested in finding out more about how they have been used, how they could be used, and what’s worked, as well as what hasn’t. The implications of the impact of this kind of resource on teaching might also be worth a look.” You can email him to find out more: Chris.Williams[AT]

We also have in mind a possible panel on digitising history – practicalities and ideas, issues and agendas, whatever. If that interests you, you might perhaps contact Tim Hitchcock (t.hitchcock[AT] for a chat before submitting a proposal.

The Metropolis on Trial
An International Conference at the Open University, Milton Keynes

10-12 July 2008

This conference heralds and celebrates the completion of the Old Bailey online project. From the early summer of 2008 it will be possible to consult at not only the Proceedings of the Old Bailey from 1674 to 1834 but also those of its successor, the Central Criminal Court, from 1834 to 1913. Papers at the conference will draw upon these proceedings, or those of similar courts in other metropolitan centres, to explore aspects of cultural, social or political life from the late eighteenth to the early twentieth centuries.

Clive Emsley, Open University
Tim Hitchcock, University of Hertfordshire
Bob Shoemaker, University of Sheffield

Proposals for papers, not more than 200 words please, should be sent by Friday 7 December 2007, to Sue Watkins, Dept of History, Open University, Milton Keynes MK7 6AA, United Kingdom. Email: S.Watkins[AT]

(I’ll be keeping this somewhere near the top of the front page for a few weeks.)

British Academy event on research outside universities

This evening event in London on 27 June may be of interest to some readers: Who’s Creating Knowledge? The challenge of non-university researchers

Is the university the primary site for the creation and authorising of knowledge? That is commonly the conventional view. But in practice large numbers of independent and non-academic researchers are enthusiastically engaged in the production and establishment of knowledge outside university walls. The panel will discuss the issues raised by the work of these often ‘invisible’ creators of knowledge, operating as they do across a wide diversity of fields of research, from family history to ornithology, astronomy to biography, philosophy to archaeology – and much else. Do such researchers present a challenge to the still often-assumed monopoly of the university over the production and validation of knowledge? Despite the obstacles they face are they perhaps following a more open route to knowledge production than in the increasingly constrained setting of university research today? Do we need to rethink the central role of the university in the establishment of knowledge? And may important new processes of knowledge-creation be emerging through the interactive potential of the internet for bypassing established university controls and evading the traditional gatekeepers to the publication and dissemination of knowledge?

Attendance is free but space is limited and you need to book a place. (And apparently there are free drinks afterwards…)

I don’t think I’ll be able to travel down and attend, but it’d be great if it could be blogged, so if anyone who doesn’t have a blog would like to go along and report on it, I’d be willing to post their reports and/or provide an open comment thread for people’s thoughts (and for links to any discussions on other blogs).