And that, in turn, got me thinking about how much freely available source material (primary and secondary) I’ve randomly stumbled across on local historical societies’ websites in the last few years. And wondering: how much more is out there?
#twitterstorians! I'm looking for local record societies – and similar history socs – which have put their past publications online as PDFs (or uploaded to the IA). I know a few already – eg https://t.co/KK6ewknqH7https://t.co/nXoqMdfLQh – but I suspect there are many more.
I got several great responses to this*, so I began looking more closely, and the TLDR; answer is: a helluva lot of it. The upshot was a Google spreadsheet, which you can see at the bottom of this post.
I’m genuinely impressed at how much stuff these societies have put online, and several more are clearly keen to follow suit – if they can fund it. Some have adopted a pragmatic policy of embargoing their most recent publications (anything between 3 and 10 years; and if you really can’t wait that long, you can buy print or digital copies – or a subscription – for sod all). They often have limited resources, and good quality digitisation isn’t cheap. So, y’know, do encourage societies of interest to you to do this; but don’t lecture them if they haven’t (you might consider instead how you could actually help them to do it).
It’s also more generally noteworthy how many societies have websites (some kept more up to date than others…), and even if they haven’t digitised the publications, nearly all have made finding aids of some sort (indexes, TOCs, abstracts, etc, even searchable databases) available.
(And undoubtedly all this applies far beyond England and Wales, but someone else will have to compile those resources. Sorry.)
Why am I telling you all this? Because these local societies (under their many and varied names: “record/historical/antiquarian/archaeological” society, or some entirely quirky local name) are treasure troves for historians, not just those who think of themselves as “local” historians. They’ve been around for a long time (many were established in the 19th century), publishing high-quality source editions, calendars, abstracts, extracts, indexes, etc, for a wide range of archive sources – parish, legal and administrative, personal, estate records, and more – as well as secondary articles. But often they were published in tiny print runs and even finding aids were hard to come by before the advent of the online catalogue. So it’s a wondrous thing that so many can now be accessed freely and located much more easily.
In addition to content found at society websites, I added a couple more tabs to the spreadsheet: some of the many publications digitised for Welsh Journals Online, and an undoubtedly tiny portion of what might be found at the Internet Archive. Enjoy exploring!
This is a version of the paper I gave at the Digital Panopticon launch conference at Liverpool in September 2017.
In the interests of fostering reproducible research in the humanities, I’ve put all the data and R code underlying this paper online on Github – details of where to find them are at the end.
Defendant speech and verdicts in the Old Bailey
Defendants’ voices are at the heart of the Digital Panopticon Voices of Authority research theme I’ve been working on with Tim Hitchcock. We know that defendants were speaking less in Old Bailey Online trials as the 19th century went on; we’ve tended to put this in the context of growing bureaucratisation and the rise of plea bargaining.
I want to think about it slightly differently in this paper though. The graph above compares conviction/acquittal for defendants who spoke and those who remained silent, in trials containing direct speech between 1781 and 1880. It suggests that for defendants themselves, their voices were a liability. This won’t surprise those who’ve read historians’ depiction of the plight that defendants found themselves in 18th-century courtrooms without defence lawyers, in the “Accused Speaks” model of the criminal trial (eg Langbein, Beattie).
But this isn’t a story of bureaucrats silencing defendants (or lawyers riding in to the rescue). I want to suggest that, once defendants had alternatives to speaking for themselves (ie, representation by lawyers and/or plea bargaining), they made the choice to fall silent because it was often in their best interests.
About the “Old Bailey Voices” Data
Brings together Old Bailey Online and Old Bailey Corpus (with some additional tagging, explained in more detail in the documentation on Github)
Combines linguistic tagging (direct speech, speaker roles) and structured trials tagging (including verdicts and sentences)
Single defendant trials only, 1781-1880
20700 trials in 227 OBO sessions
15850 of the trials contain first-person speech tagged by OBC
The Old Bailey Corpus, created by Magnus Huber, enhanced a large sample of the OBP 1720-1913 for linguistic analysis, including tagging of direct speech and tagging about speakers. [In total: 407 Proceedings, ca. 14 million spoken words, ca. 750,000 spoken words/decade.]
Trials with multiple defendants have been excluded from the dataset because of the added complexity of matching the right speaker to utterances (and they aren’t always individually named in any case). [But of course this begs the question of whether the dynamics and outcomes of multi-defendant trials might be different…]
Trial outcomes have also been simplified; if there are multiple verdicts or sentences only the most “serious” is retained. Also, for this paper I include only trials ending in guilty/not guilty verdicts, omitting a handful of ‘special verdicts’ etc.
Caveat!
Working assumption is that nearly all silent defendants do have a lawyer and the majority of defendants who speak, don’t.
Sometimes, especially in early decades, defendants had a lawyer and also spoke. Unfortunately, the OBC tagging doesn’t distinguish between prosecution and defence lawyers, and not all lawyer speech was actually reported.
But, more seriously, is it safe to assume that ‘silent’ defendants were really silent? Occasionally defendant speech was actually censored in the Proceedings (in trials where other speech was reported), eg a man on trial for seditious libel in 1822 whose defence “was of such a nature as to shock the ears of every person present, and is of course unfit for publication”. But that was a very unusual, political, case. (See t18220522-82 and Google Books, Trial of Humphrey Boyle)
[However, it was suggested in questions after the presentation that maybe the issue isn’t so much total censorship as in the case above, but that the words of convicted defendants might be more likely to be partially censored, which would problematise analyses that centre on extent and content of their words. This could be a particular problem in 1780s and 1790s; maybe less so later on.]
So work to be done here – eg, look at trials with alternative reports specifically to consider defendants’ words.
Distribution of trials by decade 1781-1880
Start with some broad context.
The number of cases peaked during the 1840s and dramatically fell in the 1850s. (Following the Criminal Justice Act 1855, many simple larceny cases were transferred to magistrates’ courts.)
Percentage of trials containing speech, annually
Percentage climbs from 1780s (in 1778 Proceedings became near-official record of court), peaks early 19th c and then after major criminal justice reforms of late 1820s swept away most of the Bloody Code, shown by red line, substantial fall in proportion of trials containing speech.
This was primarily due to increase in guilty pleas, which were previously rare. After the reforms, 2/3 of trials without speech are guilty pleas.
Conviction rates annually, including guilty pleas
(Ignore the spike around 1792, due to censorship of acquittals.) Gradual increase in conviction rates which declines again after mid 19th c.
But if we exclude guilty pleas and look only at jury trials, the pattern is rather different.
Conviction rates annually, excluding guilty pleas
Conviction rates in jury trials after the 1820s rapidly decrease – not much over 60% by end of 1870s. That’s much closer to 18th-century conviction rates (when nearly all defendants pleaded not guilty), in spite of all the transformations inside and outside the courtroom in between.
Percentage of trials in which the defendant speaks, annually
Here the green line is the Prisoners’ Counsel Act of 1836, which afforded all prisoners the right to full legal representation. But the smoothed trend line indicates that it had no significant impact on defendant speech. Defendants had, at the judge’s discretion, been permitted defence counsel to examine and cross-examine witnesses since the 1730s.Legal historians emphasise the transformative effect of the Act; but from defendants’ point of view it seems less important; for them it was already a done deal and the Bloody Code reforms were much more significant.
Defendant speech/silence and verdicts, by decade
This breaks down the first graph by decade – shows that the general pattern is consistent throughout period, though exact % and proportions do vary.
Defendant speech/silence/guilty pleas and sentences
Moreover, harsher outcomes for defendants who speak continues into sentencing. Pleading guilty (though bear in mind this only really applies to c.1830-1880, whereas silent/speaks bars are for whole period) most likely to result in imprisonment, much less likely to receive transportation (and hardly ever death) sentence. Defendants who speak are the most likely to face tougher sentences – death or transportation, more so than the silent.
(Don’t yet have actual punishments – the next big job is getting the linked Digital Panopticon life archives…)
Defendant word counts (all words spoken in a trial)
How much did defendants say? Not a lot. The largest single group of defendants is the silent (ie, 0 words). But even those who spoke usually didn’t say very much. [average overall was 55 words] Eloquent, articulate defendants few and far between!
Defendant word counts and verdicts
So if you did speak, it was better to say plenty!? Or in other words, more articulate defendants had a better chance of acquittal (though they were still slightly worse off than the silent).
Defences: average word counts and verdicts
Finish with focus on defendants’ defence statements – made by nearly all defendants who spoke and for the majority the only thing they did say (a minority questioned witnesses or made statements at other points in the trial).
overall word counts of defence statements * guilty (n=7696) average wc 44.97 * notguilty (n=1414) average wc 65.15
On average, defence statements by the acquitted were longer. Again highlights that more articulate defendants do better.
Also, there is more variety (less repetition) in the statements of acquitted defendants. 98% (1374) of their 1414 defence statements are unique (crudely measured, as text strings). Whereas 93.17% (7170) of statements by convicted defendants are unique.
Start to look more closely at what they say? Not possible yet to investigate in depth, but use some simple linguistic measures.
Defences: Words least associated with acquittal
mercy
picked
man
i
distress
carry
along
them
beg
stop
up
young
In linguistics, keywords are “items of unusual frequency in comparison with a reference corpus”. Compared the larger set of defence statements by defendants who were convicted with defence statements by defendants who were acquitted
Table above is the words least likely to be associated with acquittal – ie, the least successful defence statements…
I want to highlight:
mercy + beg
picked (+ carry might be related)
i
distress
Remember that many defence statements were not really ‘defences’; they were more of an appeal to the judges’ clemency after sentencing (‘I beg for mercy’) or claiming extenuating circumstances (‘I was in distress’) in particular. Also Playing down offence – ‘I picked up the things’.
And in general many short bare statements beginning with “I” rather than more complex narratives.
Four hopeless short defences
So I picked four of the most frequent short (non-)defences that are heavily associated with convictions, to explore a bit further. (excludes use of any of these within longer defences)
defence
frequency
% convicted
nothing to say
109
98.17
mercy
125
98.40
picked up/found
223
93.72
distress
82
97.56
Main variants:
I have nothing to say
I beg for mercy/leave it to the mercy of the court/throw myself on the mercy of the court
I picked it (them) up/found it
I was in (great) distress/I was distressed/I did it through distress
The next four graphs show the percentage of defendant speakers who use each phrase in short defence statements in each decade.
I have nothing to say
This was very popular before 1810s – peaks at use by 4% of defendants who speak in decade 1801-10 and then rapidly disappears.
I beg for mercy/leave to the mercy of the court
Slightly later popularity – slower decline after 1810s
I picked it up/found it
Less dramatic decline after 1820s.
I was in distress/did it through distress
Curious that this doesn’t appear at all in 1780s; peaks 1810s.
Conclusions
So there are variations in timing/speed of decline, but broadly, these hopeless ’non-’defence statements, which are almost certain to be followed by conviction, are all declining in use and rarely heard in the courtroom after the 1820s. That fits, it seems to me, with both the gradual decline in defendant speech and the more rapid rise from the late 1820s of plea bargaining.
First, the defence lawyer option meant that defendants were better off finding the money for a lawyer who could try to undermine the prosecution case through aggressively examining witnesses. This was happening from the 1780s onwards.
And second, the plea bargaining option from the late 1820s meant that if defendants really had no viable defence, had been caught red-handed, they were better off pleading guilty in return for a less harsh punishment.
And so: for defendants who wanted to walk free or at least lessen their punishment, if not for later historians trying to hear their voices and understand what made them tick, silence was golden.
If there’s one assumption I’d like to challenge here, it’d be that the digitisation of historical sources is all about this sort of thing:
Working on massive online history resources (Old Bailey Online, London Lives, Connected Histories, The Digital Panopticon, et al) has been keeping me occupied for the last 10 years; I very much like that state of affairs and I believe deeply in the importance of the work my colleagues and collaborators and I do at the Humanities Research Institute. But I also believe that online resources are just one facet of the digitisation of history. Our work should be a beginning, part of ongoing dialogues, adaptations and conversions, not the final word.
The good news is that both Old Bailey Online and London Lives data have been getting remixed almost as long as they’ve existed. For example, they were included in the federated search project Connected Histories, the GIS project Locating London’s Past, and in our current, massive record linkage project Digital Panopticon. And not just by us: there is the Old Bailey Corpus, and Old Bailey Online is also in the federated search sites 18thConnect and NINES. It’s also recently started attracting the attention of mathematicians and statisticians and this year has been used as a resource in a course on Scalable Data Science.
Re-use of the London Lives data outside our own domain is much less extensive, but parts of it have been used by Adam Crymble, Tim Hitchcock and Louise Falcini for a project and dataset on 18th-century vagrant lives (which we’re including in Digital Panopticon). And in fact it was their project and approach to data sharing that really got me thinking about the possibilities of remixing London Lives data on a smaller scale than our huge collaborative (and hugely funded) projects: extracting and reshaping sub-sets of data that are more manageable but nonetheless too large (tens of thousands of records rather than hundreds or low thousands) to work with entirely by hand. The London Lives Petitions Project (hereafter LLPP) is one of the results.
London Lives
London Lives is a major digital edition of a range of primary sources about eighteenth-century London, with a particular focus on the poor and crime. The project’s approach to digitisation was designed around an explicit research agenda: to foreground the lives and experiences of non-elite people and to de-emphasise institutions. More practically, the scale of the enterprise necessitated a pretty single-minded discipline to get it done. Of course we aimed to create a resource with more general usefulness, but those were the key conceptual and material factors underpinning source selection and setting the priorities for how sources would be digitised.
That meant: full text transcription followed by marking up in XML to make specific things searchable: the names of people and places, dates and occupations or social status, to facilitate nominal record linkage. And not (for example): paying detailed attention to institutional categories or structures, or cataloguing documents as archivists might do.
The result is a website (as the name suggests) that makes it easy to look for people, link together records about an individual’s life and even group related people together. But it can be harder to use for subjects that are related by other categories or themes. The keyword search is basic; there are no features to save and link, say, documents or places rather than names.
Also, the emphasis is on human judgment to make those links, and to answer questions like: what kind of person is this; how does this tagged name relate to other potentially relevant pieces of information in its vicinity? It’s been hard work to convert London Lives data for use in Digital Panopticon, which needs heavily structured name data for record linkage. So in the last couple of years we’ve been thinking a lot about ways to restructure, enhance, and build on the work we began a decade ago.
The other thing to note about London Lives is that we had to put a range of different kinds of records into a single framework, and some fitted better than others. Many of the records were bound volumes, coherent institutional products – registers, minute books, accounts, etc. A register for example is already quite structured, even when not tabular in layout; there is little ‘narrative’ language and you know what kind of info will appear where on each page.
18th-century Sessions of the Peace, presided over by magistrates, oversaw a wide range of administrative work in addition to criminal justice, including poor relief, trade and work regulations. They sat several times a year and after each meeting, the clerks would file assorted stuff from that session’s business into bundles that, ultimately, add up to a massive body of very diverse records. From three London courts (Middlesex, City of London and Westminster), London Lives has around 1250 session files (950 from the Middlesex Sessions, dwarfing all the rest) amounting to 86,000 document images, which include lists and calendars, witness examinations, petitions, court orders, accounts, and all sorts of miscellanea. (The Old Bailey Sessions Papers add another 13000 or so images but only about 20 petitions.)
Finding Petitions
Petitions are among the most common documents in those files: as it turns out, around 10,000 of them. Why are petitions interesting? The humble petition was everywhere in early modern Europe. Petitions were instigated by institutions, by groups, and by individuals, by elites and by paupers, and all sorts of people in between, direct appeals to powerful institutions or individuals to resolve a grievance or crisis. So they tell stories about lives and experiences; they aim to persuade, often to play off one source of authority against another. (work in progress bibliography)
The surviving documents are in many ways a pale shadow of the original interaction; we usually don’t know who actually wrote them, or how the voices of the petitioners might be filtered and mediated. Nonetheless, they have something to tell us about the agency of the governed and their relationships with and expectations of governments.
But also petitioners’ stories, however creative, had to conform to some formal conventions, employ certain forms of language. As a result, petitions form a potentially meaningful and findable textual corpus – if I could find the right strategies.
Just one example to underline why searching the London Lives website wouldn’t be that strategy (quite apart from scale!).
A keyword search of London Lives for ‘petition’ in Middlesex Sessions in the year 1690 returns 8 results, including 2 documents that are not petitions (although they are related). But the same keyword search and constraints in the current version of the LLPP dataset finds 11 petitions. And in total there are 66 petitions from Middlesex Sessions in that year.
search
[Confession time: I screwed up this example in the presentation; I said the total in LLPP for MiddS+1690 was 11, rather than 66. I somehow managed to forget the 11 results were only those including ‘petition’. Which is quite some difference. I thought it seemed low at the time…]
Why does the search miss so many? Many London Lives documents contain spelling variations, abbreviations, and not a few rekeying errors (which are not quite like OCR errors, but can cause similar problems for machine-readability). In fact, about one third of the LLPP petitions overall don’t contain a text string spelled ‘petition‘ at all. Others do, but only as part of a longer word (‘petitioner’, etc), which the London Lives search would only find with a wildcard search (which is unavailable at the time of writing).
I put the texts into the neat little linguists’ concordancing tool Antconc to get a wordlist, which indicates there are, in total, several hundred possible variants of words with the stem ‘petition’. In fact it’s not really as bad as that suggests, since there are a small number of particularly common forms (and often a petition text will contain slightly varying repetitions, so at least one of the common forms is likely to occur somewhere). The two endings -tion or -con will find 90-95% of petitions. So, I could handle this particular issue without too much trouble by searching with regular expressions.
But unfortunately that doesn’t deal with the problem of false positives. Many pages in the Sessions Papers that are not petitions contain ‘petition’ in some form: in fact if I simply search the entire Sessions Papers for ‘petition’ or ‘peticon’, my search will return more than 5000 pages that are not actually petitions (or in some cases, are continuation pages of multi-page petitions).
Keyword searching, extended with regular expressions, was a useful starting point for exploration, and it also highlighted just how many related documents the SPs actually contain – more than I think I’d realised. But I would obviously need a slightly smarter approach to identifying petitions.
So here’s a pretty typical petition, highlighting the formula parts of the document around the actual complaint of this petitioner. [I’ve already discussed how they work rhetorically in this earlier blog post but here I’m thinking about how they function as markers of document structure.]
Jane Browne’s petition (1691), LL: LSMSPS500100091
The example shows the elements or markers that are common to petitions (notwithstanding various minor spelling/word order variations) and aid both identification and location of start and end of the petition itself when there can be various annotations before and afterwards, including signatures:
start (1): “To The Right Honourable/Worshipful/similar title…” [After “to the”, this line can be very variable; and also it’s quite often missing or damaged]
start (2): “The humble petition of” [appears in the majority of petitions; ‘humble’ is sometimes omitted, and there can be a lot of small but annoying spelling variations]
start of the main body of petition: “Humbly Sheweth that” (again, ‘humbly’ is optional).
additionally, it’s worth noting that in the body of the text petitioners almost always refer to themselves in third person: “your (humble/poor) petitioner(s)”. ‘Humble’ and ‘humbly’ will appear somewhere along the line.
the ubiquitous ending (though again it can have quite a lot of small variations): “And your petitioner(s) (as in duty bound) shall ever pray etc“
So there’s plenty there to track them down much more reliably (and, moreover, to identify their component parts), making it possible to let the computer find the bulk of easy ‘typical’ petitions and definite ‘not’ petitions, leaving a smaller set of ‘maybes’ for more manual sifting: a few hundred, rather than several thousand.
And there are plenty of petitions that depart from the “typical” model to some degree: they might omit, or truncate, some of the expected conventions, use particularly idiosyncratic or archaic spellings, or have been penned by scribes whose handwriting was less than fluent (which is in turn likely to affect accuracy of rekeying).
Loyal readers of this blog may recognise this example:
(The phonetic “sh” spelling in petitioner is really unusual: it appears in just 8 petitions in LLPP. The entire petition is full of equally unusual spellings, and I’m pretty sure Ester wrote her own petition – the signature matches the rest – which is also very rare.)
In the end, the “non-typical” only amount to around 4-5% of petitions. But they are a little different from the rest. They skew towards the first half (and possibly the first quarter) of the 18th century (as do variant spellings of ‘petition’), and towards petitioners I’m particularly interested in, lower-status individuals and women. Not perhaps by much: women make up 20% of identifiable petitioners in the ‘typical’ 95%, and 25% in the non-typical 5%; a small number overall, but for me, doing women’s history, finding those extra 100-odd women, like Ester, is quite a big deal.
Besides, at the very beginning, it wasn’t clear just how many non-typical petitions there would be – it could have been nearer 5000 than 500 (mind you, then I’d have been looking for a different method!). But it didn’t take long to establish that they would be a relatively small number, and I do think that in a different context – if this work had been part of a much larger project working to tight deadlines – it would be a valid decision not to spend substantial amounts of time sifting manually to find those hard cases – as long as you were transparent about your methods and their limitations. But for my own purposes, and my own satisfaction, I could weigh up that choice differently – as long as I remember that, however much I’m drawn to petitions like Ester’s, they are atypical. (And I do at least know in what ways they’re atypical, and can quantify that difference.)
So, having got this data…
What Now and Next
Remix Culture? Data transformations
A key element of the project has been sharing and documenting the data and the research in progress:
Firstly, the open data contains some fairly basic metadata for the petitions and the corpus of plain text files. (This has been released in stages; I deliberately put some initially very rough work in progress out there for a couple of reasons. I’m as prone as any historian to getting bogged down in perfectionism; making public much less than perfect data is slightly painful, but creates incentives to improve it rather than keep hiding it away. I think it’s also a practical way of emphasising how data creation is a process rather than an event, and underlining the importance of versioning and documentation.)
There has also been further processing of the data for analysis (some of which will end up in the open data):
a) work on the petition texts, primarily “VARDing” and trimming. VARD is a great tool: like a spell checker, but for early modern English. It’s trainable, though I was impressed at its accuracy straight out of the box. It makes mistakes; I wouldn’t use it to “correct” transcriptions; but it’s ideal for making a more regular version of the data for textmining and quantitative analysis. VARDing was followed by stripping out annotations, signatures and so on at beginning and end of petitions, for example to enable analysis of petition lengths (nb: link is a dataviz that may be slow to load).
b) work on improving the metadata, especially
separate individuals’ petitions from institutional (especially ‘parish’) ones
using the existing London Lives name tagging to identify petitioners and start linking petitions to related records
in particular, I want to link petitions to related documents in the SPs, especially orders, so that I can examine responses – these don’t exist for all petitions but there do seem to be a lot more than I initially was aware of; as well as to related records elsewhere in London Lives, like pauper examinations .
Finally, something I’ve not really got very far with yet: identifying what petitions are about and exploring meanings. (Some early attempts at topic modelling didn’t work very well, another reason I needed to create the VARDed and trimmed version of the data.) Other sessions on textanalytics and linguistic tools at the conference gave me new ideas, although this still feels like a whole new and slightly intimidating challenge.
Concluding Thoughts
“Remixing” digitised history is something that historians do all the time, when they search online resources and copy whatever results seem relevant into their own spreadsheets and databases. But I’m not sure that they’re always doing it with the best tools for the job, or with the critical understanding they need of those resources and their limitations. Laborious “search-select-copy-paste” is fine if a resource is simply a supplement to your main sources. It becomes less appropriate if the resource is your main source, you’re using it on a large scale, or you intend to make quantitative (including implicitly quantitative) arguments based on the results. It is possible to use online search critically, but difficult without some knowledge of the underlying sources, the ability to compare different resources for the same material, and/or the time and willingness to explore different searches and methodically compare results (for a brilliant example, see Charles Upchurch, ‘Full-Text Databases and Historical Research: Cautionary Results from a Ten-Year Study’, J. Soc. Hist, 2012 [link]).
On the other hand, self-conscious digital historians (and digital humanists) are making strong critiques of online search as a methodology. “Search struggles to deal with what lies outside a set of results”, as Stephen Robertson points out. Ted Underwood argues, similarly, that “Search is a form of data mining, but a strangely focused form that only shows you what you already know to expect”.
But it seems to me that Digital Humanities-based answers to this problem often focus on the application of advanced distant reading techniques to the interpretation of Big Literary Data. I think those critiques and techniques are vitally important, but even so, the usefulness of learning how to employ them can seem rather less obvious to a social historian grappling with creating some usable research data out of digitised forms of the archival detritus of governance than to those lucky bastards screwing around with a million books (pdf). Miriam Posner has argued that what many digital humanities scholars really need, before they can get on to the fun stuff, is a lot more help with and tools for finding, cleaning and modelling data. (As she says, and Adam Crymble reminded us at the session, ‘that garbage prep work‘ is Digital Humanities too!)
The London Lives Sessions Papers can in one sense be considered big data in that there’s too much material for a person, realistically, to read all of it manually (and make sense of it). But they aren’t Big Data like a million books is Big Data. And having eventually made my dataset, I certainly want to try out that kind of analysis on the petitions texts and explore what’s possible; but I also need to do nominal record linkage and to study petitioners. My methodology for discovering petitions has been, when you get down to it, an extended kind of search. But at least for working on data creation at this sort of medium-to-biggish scale, once you’re freed from the constraints of a large database optimized for web delivery, you can get a long way screwing around with search.
The London Lives Petitions project is exploring approximately 10,000 petitions (and petitioning letters) addressed to magistrates which survive in the voluminous records of eighteenth-century London and Middlesex Sessions of the Peace which were digitised around 2008 by the London Lives project (of which I was the project manager). These documents have been difficult to access within the existing London Lives online resource because of the sheer size and variety of the Sessions Papers documents. So, the first few months of the project focused on the challenge of discovering and identifying petitions in the Sessions Papers; the resulting data, consisting of structured metadata and plain text files, has been released as open data under a Creative Commons licence. (The bulk of this effort is complete, but work is ongoing to improve the data where possible.) The data and documentation of the process can be found here.
Moving on to analysis of this new data, I’m starting from the question: What can you do with 10,000 petitions? Can large-scale ‘distant reading’ techniques tell us things that we didn’t already know from close reading of smaller, personally-crafted collections of petitions? I’m experimenting with various methods and data visualisations. But I also need to consider: what can you not do with them? Understanding what doesn’t work for data like this will be important. For one thing, the quality of the transcriptions does not match up to traditional scholarly standards: is it good enough for data mining? (This and other limitations of the original data are documented on London Lives.) With this in mind, I’ve so far done a number of mostly boring but useful things:
Processing with VARD2, a tool “designed to assist users of historical corpora in dealing with spelling variation”. This has not been intended to produce ‘better’ transcriptions (and it has probably introduced some errors along the way), but it has been very useful for dealing with common variants (eg “peticon”) and creating cleaner texts for analysis.
Identifying and removing marginal annotations and other additions that were not part of the main body of the petition texts, and some purely formula elements (like “Middx SS” at the beginning of many documents).
Breaking petitions up into their structural elements (which was important for my last post).
Additionally, as I’ve discussed in an earlier blog post, the survival of petitions (like other documents in the Sessions Papers) “could be haphazard and dependent on the preferences of individual clerks”. What is actually being counted? So, it’s necessary to put the petitions in their archival context. The Sessions Papers were loose papers relating to the work of the Sessions of the Peace (and Old Bailey from 1755), which could include petitions, examinations (from criminal, settlement or bastardy cases), calendars of prisoners and recognizances, copies of orders, lists of vagrants, coroners’ records (before they were split off into separate archives) and much besides.
London Lives Sessions Papers: image counts per year 1690-1799
The first chart above simply shows counts of the page images in the London Lives Sessions Papers, highlighting the very uneven survival of the records, especially the nine years from 1738 when very few files have survived, and many of those which did make it contain relatively few documents (or were not fit for filming). In spite of the fluctuations, however, it also indicates quite clearly the expansion of the courts’ business, especially the Middlesex Sessions (in blue), in the second half of the 18th century. (The Old Bailey series will be excluded from further analysis because it contains so few petitions.)
But while the Sessions Papers indicate ever growing business, petitions are on the decline (see also). This doesn’t necessarily mean there were fewer petitioners; it’s also possible that their petitions were less likely to be retained for long when there was so much more paper to deal with.
Petitions in Sessions Papers 1690-1799
There is also a discernible shift in certain characteristics of the petitions themselves. A large group of petitions came from parish officials concerned primarily with the administration of the poor laws – churchwardens and overseers of the poor – and I’ve been working to identify and separate these ‘parish’ petitions from petitions by individuals (and a few other institutions). Most of them relate to disputed pauper removals; smaller numbers are about poor rates assessments, negligent officials, or highway repairs. Before c.1720 these constitute no more than one-third of all petitions (in most years), but from c.1760 the figure is around two-thirds, and it’s clear that ‘other’ petitions account for most of the decline in numbers. In total, the parish petitions account for about 4600 petitions (c.46% of the total), of which about 4400 are concerned specifically with removed paupers.
Here’s a typical example of one of these petitions (1760), carefully legalistic (usually drawn up by a solicitor, with careful reference to the procedures of the laws of settlement):
The Humble petition and Appeal of the Churchwardens and Overseers of the poor of the parish of Bushton in the County of Northampton Sheweth That by Virtue of an Order under the Hands and Seals of… two of his Majestys Justices of the peace for the County of Middx… Alice Wilkinson (in the sd Order) called wife of Matthew Wilkinson (if Living) was removed and conveyed from the parish of St. Clement Danes in the said County of Middx to the said parish of Rushton as the place of the last Legal Settlement of the said Alice Wilkinson Your Petitioners Conceiving themselves aggrieved by the said Order of the said two Justices of the Peace humbly Appeal to this Court against the same…
This was the petition as the voice of early modern bureaucracy rather than ‘the people’. Comparison of average word counts for parish (pink bubbles) vs other petitions (blue) also points to the former’s highly standardised character. Overall, parish petitions are only slightly shorter than the rest, but they contain far fewer unique words.
Does a comparison of the most common words in the parish and other petitions offer any insights?
Wordle of top 100 words in parish petitions*Wordle of top 100 words in other petitions*
Word clouds may be considered harmful by some, but I think that the contrasting appearance of the two word clouds visually enhances the more prosaic table rather well: the parish petitions use a smaller range of unique words, so the top 100 are relatively evenly sized and spaced compared to the ‘other’ petitions which are dominated by a tiny number of formula words after which frequency tails off much more quickly. [*note: a small number of very common words – eg ‘a’, ‘the’, ‘for’ – have been removed from the wordle data.]
Where next? I want to start exploring that diversity more closely. I’ll be experimenting further with corpus linguistics tools and with topic modelling. And you might have noticed the bubble chart comparing parish and other petitions suggests that non-parish petitions were not simply becoming fewer in number but also substantially longer as the 18th century went on. Might this suggest that it’s primarily petitioners of lower social status who are gradually disappearing over the course of the century, leaving primarily institutions (which generated relatively short, standard petitions) and higher status individuals (creating longer, more elaborate ones)? Whatever the answer, it’s clear that tracing changes in the petitions’ language and subjects is something that I need to be investigating further.
From 1753 petition for a licence for a ‘House for Publick Entertainment of Musick and Dancing’
What does a London Lives petition look like? Well, here is a pretty typical example, from the City of London Sessions Papers (1692), in which I’ve highlighted the structural and most characteristic elements:
[1] To the right honourable the Lord Major of the Citie of London and to the right Worshipfull the Aldermen & Recorder Justices of the peace of the same Citie.
[2] The humble Peticion of the Churchwarden & Overseers of the poore of the parish of St. Michaell le Querne in London & the Parishioners & Inhabitants of the same Parish.
[3] Sheweth That upon Complaint lately made into Sir Thomas Stampe Alderman & Sir Salathiell Lovell Recorder Justices of the peace of the Citie of London (one being of the Quorum) by that parte of the parish of St. Sepulchre which is within the Citie of London, That Hannah Allen late of the said Parish of St. Michaell Le Querne aforesaid was lately come into the said parish of St. Sepulchre within the the Citie of London, & was likely to become chargeable to the same & upon oath made by the said Hannah Allen that her last legall Settlement or abode was in the said parish of St. Michaell le Querne as a hired Servant where she lived for halfe a yeare, The said Sir Thomas Stampe & Sir Selathiell Lovell by an order made their hand & Seales bearing date the 27th day of December last ordered the said Hannah Allen to bee delivered to your petrs. the Churchwardens & Overseers of the said parish of St. Michaell le Querne, & that your Petitioners should provide for her according to Law, & your petitioners have accordingly provided for her ever since.
That your Petitioners are advised that the said Hannah Allen was never legally setted in the said parish of St. Michaell Le Querne, or if so setted that she being married away from the said parish ought to bee sent to her husbands last legall Settlement.
[4] Your Petitioners therefore humbly pray your Lordshipp & Worshipps to order that the said Hannah Allen be sent back to the said parish of St. Sepulchre or that your Lordshipp & Worshipps will bee pleased to make such other order concerning her as shall bee agreable to Law & Justice
[5] And your Petitioners shall ever pray Etc.
The two middle paragraphs contain the particular narrative for this petition (or, to use the language of the petitions themselves, the ‘premises’), and they’re what most historians (including me) would most often focus on. But for the moment, I want to set that aside and highlight the common structural forms and linguistic patterns, to explore what makes this a petition, and how it works. Some of this might seem obvious, especially to historians familiar with early modern petitions, but I think it’s worth unpacking what’s going on, in the same way that Thomas Sokoll has done for pauper letters. As he says, the rhetorical elements of the letters are ‘an integral and inseparable part of the message’. (Later I’ll also look at less formal petitions/letters, but first I think we need to know something about the standard forms.)
As you might recognise if you’ve read the documentation for the data, much of what I’ve highlighted is the same language I used to search for petitions in the Sessions Papers data files, so you could say, well, duh. But I was searching for any of those elements, and others, too, and I’ve been perhaps slightly surprised at just how much the petitions conform to this very particular, formal structure (especially once various additions by court clerks have been stripped out):
[1] state to whom the petition is addressed;
[2] identify the petitioner (nearly always ‘humble’);
[3] set out the ‘premises’;
[4] detail the petitioner’s request (or ‘humble prayer’)
[5] sign off with a formal valediction which is almost always a promise to pray for the addressees (I’ll return to this)
About 90 per cent of the petitions have all these five elements. Of the remainder, most start at [2]; some end at [4]. This kind of structure isn’t just found in the London Lives archives. For comparison, consider this 1779 petition from the Massachusetts Anti-slavery and Anti-segregation petitions collection. There are minor variations in form but the similarities, despite the very different subject, are overwhelming. Indeed, while I’m not aware of printed templates (unlike some document types in the Sessions Papers), by the late 18th century petition writers could consult George Brown’s The English letter-writer for models of a ‘great Variety of Petitions on every Occurrence in human Life’.
In characterising this petition as typical, though, I’m not suggesting they were all exactly like this one. For example, the exact form of address in [1] varies a good deal. The most common, according to my database, is ‘To the Right Worshipful/Honourable his/her/their Majesty’s Justices of the peace for the County of Middlesex’. Some omit the honorifics and are simply addressed ‘to His/Her Majesty’s Justices…’ (George Brown would not have approved); sometimes individuals are named, sometimes not. (There is much more I could say about the first lines of the petitions, though I’m not sure how interesting it would be to most readers. Maybe a shorter post later.) [3] is also very frequently ‘Humbly Sheweth’ (occasionally even ‘Most Humbly Sheweth’). And so on.
On the other hand, there are elements that we’d associate with petitioning language that are largely missing from the London Lives petitions, or are much less significant. For example, the opening phrase “We, the undersigned”, which is such a familiar convention that it’s a standard template for many modern online petitions, is not a feature (even in the more usual and less memorable form “We whose names are under written”) and in its rare appearances is most likely to refer to witnesses supporting the petition’s claims rather than to petitioners (eg). And at the other end of the petition, petitioners’ signatures (marks or autographs) are quite common but far from universal.
I think the modern idea of petitions – amplified by the recent popularity of online petitioning – is very much as something democratic and populist, for which numbers and signatures are all-important.[citation needed] But that was not really the case in the eighteenth century. Yes, petitioners (pretty much by definition) were ‘speaking upwards’ and making demands of authority. But many were themselves representing positions of authority. (Almost half of the London Lives petitions are, like this example, from parish poor relief officers who are trying to get rid of paupers who’ve been foisted on them from other parishes.)
It’s not just the absence of ‘We the undersigned’ or the relative unimportance of lists of signatories. In general, first person pronouns are very uncommon in the London Lives petitions and, if anything, ‘We’ seems to be even less common than ‘I’. (This is based on some simple searches; I need to investigate more systematically for more certainty.) Instead, petitioners nearly always refer to themselves as ‘your petitioner(s)’. It seems to me that this highlights that what really matters here is not the expression of the ‘voice of the people’ but the opinion of the authority to whom the petition is being addressed.
The language of deference in these petitions goes deep and far beyond the obvious and liberal use of ‘humbl(e|y)’. And yet they are nonetheless artefacts of some kind of political agency: people making demands of authority. Petitions to the Bench of magistrates may be deeply deferential but it shouldn’t be forgotten that in many cases the petitioners are explicitly challenging other figures of authority: parish officials, individual magistrates, gaolers and workhouse masters, employers, husbands.
So I think it’s helpful to think of early modern petitions, in terms of politeness theory and face-threatening acts (FTAs): the rhetoric of deference and submission in petitions can be seen as a way of mitigating the potential threat contained in social inferiors’ demands. [There seems to have been rather a lot written around this since I read Brown and Levinson back in about 1999 for my MA thesis, so I have some catching up to do and my thoughts may seem a bit, errm, naive to linguists. Be kind!]
With that in mind, I want to finish by looking more closely at my model petition’s element [5]. These closing phrases have been intriguing me because they are very common but don’t have an immediately obvious function in quite the same way as the rest of the petition. The request [4] normally contains plenty of humbleness and deference, and about 8% of petitions do in fact stop there. So [5] is clearly not mandatory, but it’s nonetheless present in around 9/10 of the petitions. (Plus, a good chunk of the remaining 2% contain other sign-offs like ‘your obedient humble servant’, or the end of the petition is illegible due to the poor condition of the original.) So what is it actually doing?
Like the other elements, it does vary in details. It’s very commonly expanded to ‘And your petitioner as in duty bound shall ever pray [etc]’; conversely it’s often simply ‘And your petitioner shall pray [etc]’ and occasionally can be as abbreviated as ‘And etc‘ (!). ‘Will’ is also used instead of ‘shall’ (though I think ‘shall’ is more common, and more forceful in meaning.) More unusual additions make more explicit what may be encompassed in that also optional but very frequent ‘etc’, eg:
and for this yours honers goodness and Cllemincy as in all humblle duty bound this poor person will ever pray with a grateful acknowlledgment and thankful hart.
And your Petitioner as in Duty bound will Ever pray for and am your Worships most Obedient Humble Servant at Command
It’s doing a lot of hard work, this phrase. The first thing of note (which the modern secular reader might easily forget) is that it’s an expression of piety, of assumed shared religious values (I will pray). Second, it’s one last statement of submission towards the addressee (as in duty bound, your obedient humble servant, I will pray for you). Third, it’s an expression (pre-emptively) of gratitude – normally implicitly and sometimes more explicitly, as in the “grateful acknowlledgment and thankful hart” example. Discussing how expressions of gratitude functioned as politeness markers in early modern English, Mattias Jacobsson notes how they can be intensified by expressions of deference (including the ubiquitous modifier ‘humbly’). ‘Compound thanks’ are rare in the corpus analysed by Jacobsson (the Corpus of English Dialogues), but petitions are a rather different genre, in which a helluva lot of gratitude and deference may be needed to mitigate the FTA of the petitioner’s demands.
One final note, though: all this gratitude and deference is conditional, as is made clearer in examples like these:
and in So Doing yr petitioners and their Families as in Duty Bound Shall Ever pray Etc
and for this yours honers goodness and Cllemincy as in all humblle duty bound this poor person will ever pray with a grateful acknowlledgment and thankful hart.
most humbly Implores that this Court will adhear & listen to this Application and shorten the time of his sentence in Prison according as they of their Great Wisdom shall seem Meet which if so happy as to be granted Petitioner will as in Duty bound Ever pray and retain a greatfull Acknowledgement for such their Mercey & Clemency shewen to this his Humble Petition
While paupers certainly claimed to be humble, grateful, and sorry in their approaches to the local state, in practice it is striking how often the poor asserted their ‘rights’ (moral, legal, or christian) and the ‘duty’ of the parish.
The petitioner’s gratitude can’t be taken for granted; submissiveness and humbleness (however often reasserted) are strategic. Petitions, even where we can’t be sure that they were ‘authentic voices of the people’, werenegotiating tools.
…..
A housekeeping note: since the last post, I’ve done a further update to the petitions data, in which I’ve cleaned out more non-petitions (it’s now down to 10,045) – but I think I have found them all now…! As a result, I’ll need to check the numbers and update the visualisations in my last post at some point.
Since my last post introducing the new London Lives petitions project, I’ve released a slightly updated version of the data: I added some petitions and letters I’d missed on the first sweep and removed a few documents that were either not petitions or were so fragmentary that they were not worth retaining. There are now 10,187 petitions and petitioning letters. (You can download the latest version at the project’s github page.)
In addition, the project now also has a homepage at my own website, where you’ll find a growing number of data visualisations and resources (and sooner or later there’ll be some sort of search facility). The visualisations I’ve done so far present some fairly basic counting, aimed at getting an overview of the petitions over the period 1690-1799, where there are gaps, whether there are any interesting patterns that point to questions worth closer investigation, and so on. (In most cases, I’ve left out the handful of petitions from the Old Bailey Sessions Papers.)
This one, for example, aims to give some sense of how the petitions fit into the Sessions Papers as a whole.
Petitions in London Lives Sessions Papers (City, Middlesex and Westminster), 1690-1799
(I should note that, while I have an accurate count of documents in the petitions subset, it’s hard to ascertain that precisely for the rest, because some documents have multiple pages but London Lives doesn’t have any markup for that, only for images. The markup does contain a tag (<pb>) to indicate where text spans two pages, but that isn’t quite the same thing, because it’s not used where a page break coincides with a paragraph break. So the count for the larger body of documents will inevitably be slightly too high. However, I think it should be close enough, and consistent enough, to serve this purpose.)
A few things jump out from this graph: firstly, there is a noticeable peak in petitions in the mid-1710s and an equally noticeable decline from around 1740, especially relative to the overall business of the courts. Secondly, there are hardly any petitions for almost a decade from the late 1730s. On breaking down numbers by court, it’s clear that this is due to significant gaps in the Middlesex series of Sessions Papers, which accounts for around 3/4 of the total petitions. The Middlesex SPs during that period survive patchily, and those that do survive contain relatively few documents, and almost no petitions at all. Also, most of the Middlesex files that are missing from London Lives because the LMA archivists deemed them “unfit” for microfilming are from the same period.
London Lives Petitions, by court, 1690-1799
So it’s clear that the gaps are all about problems with record survival. The Sessions Papers were not a record of the court’s formal business (like indictments or books of court orders) or of financial commitments (like recognizances). They were really just random Stuff™. So their chances of long-term survival could be haphazard and dependent on the preferences of individual clerks. As I’ve already mentioned, some petitions have survived only in fragments (example), and the poor condition of some documents is a reminder that the bundles of papers were not always kept in ideal conditions.
For the same reason, I’m wary of reading too much at this stage into the apparently exceptionally busy petitioning years 1715-17, but it does suggest they’ll be worth a closer look, especially as this was just after a period of regime change and some political turbulence. On the other hand, I do think the fall in petitions’ numbers (and even more so as a proportion of the whole) from mid-century is sustained enough to be significant, if not necessarily straightforward to explain. Perhaps there really were fewer petitions, or perhaps clerks of the peace thought them less valuable and were less likely to keep them. (It also suggests that I may need to explore what’s in the rest of the Sessions Papers, a much larger undertaking that I didn’t really want to try at this stage!)
The other interesting counting exercise has been to measure the length of petitions, using a scatter plot for word counts similar to one previously used for Old Bailey Online trial lengths. (The first iteration of this exercise was incidentally very useful in identifying fragments and short non-petitions and the graphs now on the website represent a second attempt.) Because there is a dot on the graph for each petition it also shows quite neatly the fluctuating numbers of petitions. While the bulk of the petitions were less than 500 words in length, there were small numbers of much longer petitions, so there are two versions of the same graph. The original one uses a linear scale, which I find useful for seeing the overall range; the second (below) a logarithmic scale, which gives a more detailed picture of the range within which most of the petitions fall, making it clearer that in the second half of the period, there are fewer short petitions.
Length of London Lives Petitions (by word count), 1690-1799
Does that account for the smaller numbers of petitions overall? Perhaps the clerks were choosing to keep only the more substantial petitions. But then again, it might be a sign that petitions were becoming (even) more formalised, less likely to be used by individuals than institutions. The longest petition in the dataset (1793), at nearly 3400 words, is by an official and concerns a breach of building regulations.
So, next on the agenda is to start exploring the actual content of the petitions!
Note
The visualisations so far have been done with d3plus, an extension to the D3.js javascript dataviz library. D3 is amazingly powerful but really hard to learn and use, and d3plus has made the learning curve much less steep (though you’ll still have to work out how to get your data into the right shape). I’m also trying out dimple, another “D3 for Javascript Dummies” resource.
I’ve spent the last couple of months on a mission to find petitions in the Sessions Papers of London Lives. The outcome of that quest is just over 10,000 petitions which I’ve made available under a Creative Commons licence, with full documentation, on github:
But this is hopefully just the beginning of a project to explore the London Lives petitions in more depth. I’ve long been a fan of the humble early modern petition addressed to local magistrates, documents to be found in vast numbers in local and legal archives. Petitions were instigated by institutions and by individuals, by elites and by paupers, and all sorts of people in between. They were used by convicted criminals to beg for the royal pardon; by officials and contractors to claim expenses for government work; by private individuals to complain about abusive behaviour by neighbours, employers, apprentices, husbands or local officials; by parishes to appeal official decisions about paupers and vagrants; to claim exemptions from local office or taxes; and more.
Petitions like the one above, addressed to Middlesex magistrates in 1715 by Ester Cutler, a ‘Weddow woman’ with ‘nothing to live apon but what she can gitt out of selling a few herbs’, and requesting an exemption from paying poor rates. Ester Cutler’s petition is perhaps unusual in some ways. Apart from the typically deferential language of “your humble petishnor”, it does not quite follow many of the formal conventions of petitions in the Sessions Papers, to be seen in the petition below, from the parish officers of St George Botolph Lane in 1693, which has a much more elaborate opening address (‘To the Right Honoble. Sr . John Fleete Knt Lord Maior of the City of London…’) and the usual line ‘And they shall Pray Etc’. This type of petition is much more formulaic and generally adheres to more elaborate d structural and linguistic conventions.
Ester’s spelling is also unusual: in the 10,000 London Lives transcriptions, I can find just one other petitioner who writes ‘petition(er)’ with the spelling ‘sh’. The spellings ‘gentellmen’, ‘consederacion’, ‘weddow’, ‘apon’, ‘gitt’, ‘desiers’ are all equally unusual, or nearly so. The spelling of even high status petitions is much less standardised than in modern writing (and full of abbreviations), but by the early 18th century their variations are smaller – ‘poor’ vs poore’, ‘relief’ vs ‘releif’, etc.
It looks as though Ester wrote her own petition (the hand including signature is the same throughout), and that too is probably quite uncommon (though another question that warrants further investigation). Ester’s writing ability, for all her non-standard spellings, suggests that even if she really was as poor as she claimed to be in 1715, she probably came from more affluent origins. Perhaps her claims of poverty should not quite be taken at face value; she’s a rate payer, not a pauper. But she might well have been on a social descent towards that fate, and that was an experience of ageing and widowhood that was far from unusual in 18th-century London. Widows like Ester were frequent petitioners.
In 1713, Mary Donne of St Giles Cripplegate was another petitioning for exemption from parish rates. She too emphasised her continuing efforts to make ends meet but
now being reduced to great Poverty by reason of her Age and hard Labour, Whith the loss of one of her Eyes, and a great dimness in the other, that she is uncapable of doing any manner of Work for a livelyhood except Knitting.
And Rachell White, the widow of a London tradesman, petitioned the magistrates in 1721 at ‘about 70 years of age and by Long Sickness very infirme’, to complain about her treatment by local officials and demand the restitution of a parish allowance which enabled her to live in the ‘Coopers alms house’ (with the warning that if she could no longer stay there she would become a much greater expense for the parish). The magistrates ordered the parish to pay her a weekly allowance of 1 shilling.
Wherefore yor. poor petitionr most humbly Implores yor Honrs in yor Tender Compassion & timely Consideration; would be pleased To order the said parish to allow her a Competent mentanance The Small allowance She had being taken away by Mr. Masters the present Churchwarden; his pretence being that she doth not want any parish allowance, by Reason She is att present in the Coopers alms house; The Master with Sr. Peter Eton who put her in Seeing her great poverty will Certainly turn her out ffor they Say they will not Suffer any their to be Starved to Death without the parish she belongs unto will allow her a mentainance She Shall not be their to Dissgrace the Company but must be wholy put upon the parish wherefore She most humbly Beseeches yor Honours timely Relief.
A very quick post to note that I have an article in this volume, based on my presentation at the first Our Criminal Past event in 2013. But there’s plenty more there for crime historians to be interested in.
