Monthly Archives: January 2015

0.2 is live!

I’ve just put the new version of the Party Funding database up and it appears to be working. I’ve added a lot of new features to it.

The biggest change is that there are now pages for “recipients” (the term I’m using for any organisation or individual who is listed on the register receiving funds including political parties, MPs, referendum campaigns, and so on). This is fully integrated into the other pages, meaning you can search for them directly, or you can look up a funder, click on the political party they’ve donated to, and look at that as well.

The biggest issue for the recipient pages was how to deal with all the data. With the exception of the biggest trade unions, most funder pages don’t have that many entries on them. By contrast, the Labour Party has reported over 14,000 individual contributions (and counting). I’ve dealt with this in two ways: firstly by introducing charts, to give all the data a more visual appeal. Secondly, I’ve made the tables load separately so you won’t have to wait staring at a blank screen until all the donations upload. I want to improve this further, and in particular need to sort out the search page which is currently a little sluggish, but it is running much more smoothly than it was.

The charts are something I would especially like feedback on. Currently, I’ve set it up so you can easily switch between charts showing amounts received by what the Electoral Commission terms ‘donor types’, but there’s a wide range of different ways I could present this data, such as how much is by cash as opposed to in kind, amounts received by each accounting unit and even by certain user generated tags (which should come on stream in the next update). I’d like to hear what people would find most useful.

The other minor change I’ve made, following my last blog post, is to include pages for each individual entry on the register so you can easily compare my data against the Electoral Commission’s to ensure it is accurate. Hopefully this will alleviate some people’s concerns that I’m just making these figures up!

Anyway, onto the next upgrade, which will massively open up the scope for user-generated content.

Ex Machina film

Testing the machine

So, this evening I went to see the marvellous and challenging film Ex Machina, which is about artificial intelligence, power and abuse – I strongly recommend it.

While I was watching the film, I experienced my first bit of controversy with the Party Funding database. Someone listed as a donor on the site claimed that our records were incorrect and demanded a correction:

A little background: Rupert Read is the Green Party Prospective Parliamentary Candidate for Cambridge City. He’s recently been embroiled in a bit of controversy over some comments made about transgender people, which he has subsequently now apologised for. One blogger, Zoe Stavri, used this website to point out that he was a major donor to the Green Party, which is how, I presume, Rupert Read came across this site (full disclosure: Zoe is a personal friend of mine and I’m grateful to her for helping to promote the site; I also know Rupert, largely though my previous work at the New Politics Network and Unlock Democracy).

It was then suggested that there are other discrepancies on the site, involving other Green Party donors. Specifically:

This is potentially a big problem: if the website is showing erroneous data, that is a very big deal. Fortunately, I have checked with the Electoral Commission database, and it would appear that this is not the case. I can’t link directly to the Commission’s database (nor can it be used on mobile devices!), but the following are screenshots showing what the Electoral Commission themselves have down for Rupert Read and Noel Gillian Kirkby:

Rupert Read donations from the Electoral Commission website Notel Gillian Kirkby donation details from the Electoral Commission website Noel Gillian Kirby donations details from the Electoral Commission website


(Note: there are two screenshots for Noel Gillian Kirkby and Noel Gillian Kirby – I’ve merged these two on the Party Funding database because they are presumably the same person and no-one has yet challenged this assertion).

As I hope you can see, the data on both the Commission database and my database are identical. This does not mean of course that either Rupert Read or Noel Gillian Kirkby did indeed make those donations, merely that that is what the Electoral Commission’s register says. I make no claims as to the accuracy of this data; indeed, the whole point of this project is to subject it to greater scrutiny than I believe it currently enjoys.

It could be that the Electoral Commission has made a mistake. My understanding however is that since the mid-2000s, the political parties enter the data directly themselves via the Electoral Commission’s own e-filing system. On the face of it therefore, the data came from the Green Party directly. I suggest to Rupert Read that he begins his investigation into how he came to be listed at the Green’s ninth biggest donor of all time (and third biggest in 2014) with his own regional party.

Lest you think I’m complacent about this little episode, I can assure you that I’m not. This experience has thrown up a whole number of issues which I need to contemplate.

The first is that while the data presented here is to the best of my knowledge accurate, it isn’t as transparent as it could be. I recognise now that I should have created pages for each individual donation containing all the data provided so that they can be properly scrutinised directly. I’m going to include that in the next build (which is coming along very nicely, thank you). The second is that aside from emailing me, I haven’t developed any kind of automated mechanism to allow people to flag up what they believe to be questionable data. I have no intention of changing any data here that hasn’t been corrected by the Electoral Commission itself, but I think it is reasonable to allow people to flag concerns on the website if they feel they have been misrepresented. I’m already planning a commenting system, but I’m now considering whether a “red flag” system is needed alongside that.

The third issue is more troubling. Because this data is taken from the Commission website at a certain point in time, I don’t have a way of correcting old records if they are subsequently changed. I’m all set up to add new quarterly data as it is published, but there is no way I can correct older data, other than by uploading the entire dataset periodically. Currently, this is the only solution I have, but it is a time consuming one. Fortunately, I’ve designed the system so that if I do reload the original Commission data periodically, it won’t stop the rest of the website working (in principle, at least). But I’d rather be able to simply update any changes on an ad hoc basis.

To summarise: I’m pleased to say that the database has passed its first real test, but even then I’ve learned a lot from this experience and it has caused me to rethink how it should work. It’s positive then, but it involves more work and might delay the release of 0.2 by a little.

A Block for the Wigs, Gillray

A quick update

I launched the website yesterday and have been really pleased with the reception it has enjoyed thus far. Indeed, I was hoping for a little more critical feedback, so don’t hold back! I’m really interested in knowing what people would like to see this website do.

I haven’t analysed the visitor data in great depth thus far, although I’ve noticed a lot of people using the names of political parties in the search bar. You won’t find them right now, sorry; you’ll have to make do with the top funders tool for now. But the party, party accounting unit, regulated donee, third party and permitted participant (ugh! so much jargon) pages are coming soon – indeed, I’m hoping to make some progress on them today.

You might notice a small change with the search bar; I’ve slightly resized it and added a list of “currently popular” funders. This is to help visitors navigate around the site. Currently, the top choices are Mr George G Watson who by dint of his £950,000 donation to the Lib Dems last August puts him on the top of the top funders tool, and Lord Matthew Oakeshott, the former Lib Dem peer who it was revealed yesterday has divided £600,000 between 30 Labour and 15 Lib Dem candidates to fight the next election with (presumably these donations will appear in either the next set of figures to be published by the Electoral Commission, or the Jan-Mar 2015 register which will be published just before the election itself).

The algorithm I’ve used for the “popular” list is based on the idea of today’s visits counting as worth double the visits yesterday, which in turn are worth double the visits from two days ago, and so on for the next seven days. I’ll be keeping an eye on how well this seems to be reflecting actual visitor behaviour, so it may change again in the future, but it will do for now.

Anyway, today’s task is to upload the register of political parties and construct similar tables of “regulated donees” (elected politicians and partisan organisations). They don’t publish a register of these, but as far as I can see they are on the same database that the Electoral Commission uses as parties; they certainly use the same unique identifiers. Once that is done, I just have to stitch it together with the donations register and we should have a basic recipients page up and running.

Thanks again for your interest in this project.

Monopoly Board

Version 0.1 is now live

Don’t consider this to be a finished website, but I think I’ve finally reached a stage when I wouldn’t be too embarrassed about releasing this thing into the wild. Visit the database here.

I started work on this website at the beginning of January, but it is really the culmination of years of work at Unlock Democracy and its predecessor the New Politics Network. Eleven years ago, the NPN was responsible for the first attempt to build a public-oriented site based on the Electoral Commission‘s register of donations, in the form of Cleanpolitix. It was pretty basic and incredibly frustrating to use, so I redeveloped it as Party Watch.

Party Watch was useable but pretty ugly and I made a number of mistakes along the way. That project ended in 2007 and I moved onto other things. But I was always keen to revive it and implement what I had learnt from the experience.

For various reasons, UD could never prioritise it as a project, and I never did get back to it. The investment it needed wasn’t finances however (well, not much), but time.

I’m currently out of work, and thought this would be a good project to start – both as a way to scratch my itch, and to develop my own coding skills. So this website has been my first experience at using jQuery, Bootstrap and Object Oriented Programming (which I’m slowly getting my head around).

A technical challenge

As you will quickly see, the website doesn’t do an awful lot right now. Currently, it lets you look up party funders and see who they have given money to – and when. The biggest part of my job thus far – literally 95% – has been developing a system for finding multiple records of the same organisation or individual and merging them.

This matters because, as anyone who is vaguely familiar with the Electoral Commission’s register will be able to tell you, it’s an unholy mess. Party funders do not register, the parties do, and it is the parties that submit the data to the Commission. This arrives in a variety of different formats. Thus it is that the same individual might be listed as “John Smith”, “John A Smith”, “Mr John Smith”, “na John Smith”, “Dr John A Smith”, “Sir John Smith” (once they get an honour), and any variation therein. And that’s not counting the typos or the even wider variety of ways in which company and other organisational data gets submitted.

This messiness is ultimately what killed Party Watch – keeping ahead of all the different misspellings and misappropriations defeated me. So Party Funding has been built from the ground up with this in mind. And it has been designed to be reversible – one of the big mistakes I made with Party Watch – so that if I make a mistake it can be undone. Sadly, that means having the database comparing three different tables of data, which slows it down somewhat.

I’m not even close to deduping all 45,000+ records on the database. However, I’ve made it a relatively simple task. And you can help me with this! Simply register on the website, search for someone and, if you come across two or more records that should clearly be combined, combine them by following the instructions on the tool. You should hopefully find it pretty simple.

Where next?

After a short break, my next project will be building pages dedicated to the recipients of funds – political parties, their various accounting units (local parties, mostly), third parties (campaign organisations which campaign in elections), “permitted participants” (campaign organisations which take part in referendums) and “regulated donees” (political organisations and politicians). The data on this side of the table is much easier to handle because the recipients are registered and have unique identifiers. On the other hand, there is much more data to handle (try selecting the top donors of all time for the Conservative Party to give you an idea), and presenting this in a meaningful way that doesn’t the server to slow down to a crawl is my next big technical challenge. It should be fun!


Please submit all substantial feedback and advice on the relevant page for ease of reference. If you’d rather communicate by email, drop me a line on jamesgraham*at*partyfunding*dot*uk. Thanks!