Owen Jones: And How He Gets Away With It

On Thursday morning, I received an email from a stranger, Henry, who had found something in Owen Jones' bestselling new book The Establishment: And How They Get Away With It. I've previously asked Jones about the way he's presented a quote in the book, and got what I thought was a very unsatisfactory answer. Henry highlighted a claim Jones made in The Establishment, that 92 percent of the top 50 publicly traded companies in the UK had a parliamentarian as either a shareholder or director. But the report Jones cited for this said it was 46 percent of the top 50 - Jones had mistaken this for 46 out of 50.

This is a howler, but hadn't he just misread a figure? Well, no. Looking at it closely, I saw Henry's point: this seemingly trivial error was more serious than it appeared. Jones didn't simply misread a figure in the source he'd cited - he didn't research it at all, and even as 46 percent it's very misleading. The figure also happened to be one Jones had seized on and mentioned in several interviews and articles, and always as if it applied to today, when his cited source was a 2012 report. His repeated mentions of this statistic were widely noticed - for example, one tweet by Bonnie Greer about it during a talk he gave at the LSE last week was retweeted nearly 600 times:

'92% of top 50 companies in #UK have a director or a shareholder who is also-an MP via @OwenJones84 #LSEestablishment #HoC #transparency'

One of the replies Greer received to that tweet was 'god, that's terrifying!' And it is.

It might well be that the relationship between MPs and business is too cosy, but this statistic is incorrect. What's worse, though, is what it says about how Jones researches. In the book, he introduced the figure by saying 'according to a 2012 study', and endnotes page 290 of this report by Democratic Audit UK. Jones' implication is clear: this data is from 2012. But if you look at page 290 of that study, it reads:

The hyperlink in blue should stand out for any researcher, for several reasons. The first is the date in brackets: 2006. So straight away you can see this isn't even up-to-date information for 2012, but something dating back nearly a decade. How useful can it be to apply now? Mightn't a more up-to-date figure be more meaningful than this? Following on from that, if you do use this information, it is misleading to say 'According to a 2012 study'. They've simply mentioned it. What if they had been discussing data from 1932? Would it still be acceptable to present it as 'According to a 2012 study' with no indication of the year the data actually applies to? Of course not. It isn't here, either.

And why use this study at all? The source is right there, in a hyperlink. If you click it, you come through to Mara Faccio's study. And now several problems present themselves. The first is the date at the top of it: July 13, 2004. This means Democratic Audit have been sloppy, because they've misattributed it to 2006 when the research was in fact published in 2004. It might be a small error, but this sort of Chinese whispers is precisely why researchers look at primary sources.

And when you look at Faccio's study, all sorts of problems present themselves. The column in the table that shows the 46 percent figure for the UK has the heading '% of top 50 firms connected with a minister or MP, or a close relationship'. Faccio's definition of 'MP' includes members of the House of Lords. What does she mean by 'connected with' a minister or MP?

'I say a company is connected with a politician if one of the company’s large shareholders or top directors is: (1) a member of parliament, (2) a minister or the head of state, or (3) is closely related to a top official.'

Her definition of a large shareholder is anyone controlling at least 10% of a company's voting stock, and this is referred to as simply 'shareholder' elsewhere in her study. So her definitions are already broadening this out hugely. How does she define 'top official', and what does she mean by 'a close relationship'?

'Connections in this case occur when a large shareholder or a top director is: a friend of a minister or MP; a former minister; currently a politician, but has left the business after 1997; a foreign politician; or is a person known to be associated with a political party.'

This is extremely broad indeed. Even if you remove the 92% howler in Jones' book, what we're left with is not that 'according to a 2012 study', 46% of the UK's top 50 companies had a British parliamentarian as either a director or shareholder. It's more like: according to a 2004 study, 23 of the top 50 companies in the UK had a director or significant stakeholder who was either a British parliamentarian, someone closely related to a top official, someone connected to any of the above by friendship, or someone known to be associated with a political party. And even that isn't quite as broad as Faccio has it.

It's a pretty useless statistic, I think. But what is telling here is that Jones clearly hasn't consulted the primary source, which was hardly difficult to find: it's hyperlinked in the study he cites. Even more telling is how he reacted when I pointed this out to him on Twitter. He said a lot of sensible stuff. He tweeted 'I take accuracy very seriously and will immediately correct any errors made in good faith' and 'I take stats and accuracy very seriously so welcome any corrections'. Excellent.

And superficially it seems he did in fact do this. He said the 92% figure had already been corrected in the book, and when I pointed out it was in my ebook, which I'd downloaded that morning, he replied 'glad you've told me that - it's been corrected for print, I didn't know that it hasn't for ebook, so thanks'. And he wrote a blogpost seemingly correcting the error. I say seemingly, because all he's done is called it a typo and corrected the 92% to 46%. I repeatedly pointed out these other problems to him, and asked him if he had read Faccio's 2004 study, which is where the data comes from. He didn't reply to that question. So he didn't bother to look at the source when he wrote the book. And when it was pointed out to him, he still didn't bother.

Or perhaps it's even worse than that. He did bother, but just doesn't want to admit that the information is out of date. That sounds like quite a charge, but look at how he has worded the 'correction' on his blog.

'In my chapter on ‘The Westminster Cartel’, I quote from research about the links between MPs and the corporate world discussed in the 2012 Democratic Audit report.'

Discussed. Yes, it's discussed. But when is the data from? 2004. He doesn't mention this. He also writes:

'Also, here’s a graph (I like graphs) to put it in perspective:'

And pastes in the table from the 2012 report. I like graphs and tables, too, if they're accurate. But if he'd bothered to read Faccio's report - which, surely, by now, having had this pointed out, would be the thing to do - he'd have realised that Democratic Audit's table is wrong. The column heading is '% top 50 firms connected with a minister or MP'. But Faccio's definition of this was much broader, and her table for the same percentage figure included the words 'or a close relationship'. Democratic Audit's report says Faccio 'found that 46 per cent of the top 50 publicly traded firms had a British MP as a director or shareholder'. This simply isn't what she found. So their use of sources is very sloppy. As is Jones'. He included Democratic Audit's table as if that in itself showed how even without his 'typo' his argument held, but he didn't mention that the data is from 2004, includes peers, top officials, friends of all of those, people associated with political parties and so on. He still hasn't looked at the primary source. 

Neither is this the only problem in Jones' book. My emailer, Henry, pointed out other problems with his research - he seems to believe the only social security spending that goes to unemployed people is the Jobseeker's Allowance - and I'd already asked Jones about a quote in the book, to which he gave a very unsatisfactory answer. So? None of this is going to change the universe. Ebola's on the loose. Putin is persecuting minorities and invading places. Islamist extremists are beheading aid workers. Why do I care? Well, I care about more than one thing, and care about them in different ways. If you want to throw out whataboutery or thatmuchology, do, but it's not a great counter-argument to the points I'm discussing.

Neither is attacking me for pointing this out. Someone on Twitter asked me yesterday why I'm 'so keen to defend state and corporate power' against Jones. I'm not. This isn't politically motivated.  I broadly agree with much of Jones' politics and have enjoyed several of his pieces. Neither do I have any personal animosity towards him - I don't know him. This is about journalistic ethics. I also take accuracy seriously, and integrity. I think it's quite possible that the connection between political and corporate interests in the UK is too cosy - but I'd like that argument to rest on solid ground. And as Jones seems not to know the very basics of how to research information, how much of his book can I trust? Worse, as he still hasn't understood the errors and looked into it properly once it's been pointed out, how can I take that he's making his arguments in good faith? It's bad enough he didn't research this properly for the book, but much worse not to bother once it's been brought to his attention. Is his omission of the primary source, 2004 and all the other provisos in his 'correction' because he's lazy - or because he doesn't want to admit that his entire argument using this source is incredibly weak and in fact should probably be withdrawn entirely?

This morning, I received another email from Henry. I asked him if I could reproduce it here, and he agreed, so here it is, with a couple of typos cleaned up (actual ones, not Owen Jones-style ones):

'Hi Jeremy,

I can't help but keep looking into Owen's book. If you're all done with this story, and I would totally understand if you were, then please just let me know and I won't email you about it anymore. But I found another outrageous example that really illustrates Owen's inability to do proper research. If you don't want to know, just stop reading here, because this is a bit long, but it's also a very clear cut-case of what might be some of the sloppiest research I've seen in a major book. I also noticed that his book has been on the Sunday Times bestsellers chart for three or four weeks now, which just makes this all the more frustrating. If you're interested, read on, if not, stop here and just let me know you're walking away. I can always try sending this to someone else and see if they can take it up. I saw that it got some coverage at order-order.com after your tweets and I can send it to them if you're not interested and they might run it. And if you want to use my first name to let people know that I'm the one that's become a bit fixated on this and it's not just you, feel free. Please just don't post my email address or full name.

Okay, here goes. I have the ebook version of his book and decided to just check sources that are available online. I picked Chapter 9, "Scrounging off the State" because I figured there would be lots of stats thrown around in there. I went to the source page for that chapter to look for online sources. The first online source I didn't check because it looked straightforward enough and was just a quote, not a stat. The next online source listed was the sixth cited source for the chapter and had quite a few figures being cited in the text. Here's what I found:

From Chapter 9 - Scrounging off the State

Here’s Owen Jones:

'Perhaps nothing encapsulates state-subsidized capitalism like the UK arms industry. The Campaign Against the Arms Trade (CAAT) has examined the subsidies showered on the sector on an annual basis. These include £16 million for the Defence Export Services Organisation, which acts as an official salesman for the industry; £24 million for defence attachés and embassy staff to promote military exports; and £670 million on research and development. The CAAT found that, in total, arms exports received a government subsidy worth £890 million each year, or £13,000 per export job.'

The source for these stats: http://www.caat.org.uk/resources/publications/economics/subsidies-factsheet-0504.php

Firstly, the Defence Export Services Organisation doesn’t exist anymore! It was scrapped by Gordon Brown in 2007! 7 years ago! And yet he’s presenting it as if they’re still pulling in 16 mil a year. Notice he refers to them in the present tense.

Here’s The Guardian and the Telegraph on the closure back in '07:

http://www.theguardian.com/business/2007/jul/26/1
http://www.telegraph.co.uk/finance/markets/2812952/Fury-as-DESO-is-scrapped.html

And here is CAAT on where the 16m figure is from: 'DESO's forecasted net operating cost for 2003-2004 is £16m.' So it was a forecasted figure from 2003-2004 anyway. Not that it matters since DESO don’t even exist anymore. But why such old figures? His source for this whole paragraph, the CAAT factsheet, was written in 2004 and has never been updated. So all of these numbers, not the just ones for DESO, are at least a decade old, some of them older and some of them just made up, as I'll explain. Not that you’d know this from how he’s presented the information in his book.

Secondly, that “£24 million for defence attachés and embassy staff to promote military exports”? CAAT basically just made that figure up. If you read their out-of-date factsheet they admit to using estimates and “guesswork” where there are no official figures available. And their “guesswork” is godawful from a research perspective.

Here’s how CAAT very clearly explains in that factsheet (the one Owen read and cited!) how they arrived at their guesstimate of £24m:

'A 1989 National Audit Office survey estimated that forty percent of a defence attaché's time was spend promoting military exports. As post Cold War oversupply in the arms market has led to a buyers' market, and because the number of markets has increased, it seems highly unlikely that this element of the defence attaché's role has decreased. The cost of defence attachés for 2002-2003 has been given as £37.1m giving an export subsidy of £15m. There is far less relevant information available for embassies as a whole. It has been reported that the former UK Ambassador to Iran in the 1970s spent the 'vast majority' of his time trying to sell weapons to the Shah. While that may have been a one-off situation, it seems likely that a significant proportion of the effort of embassies in, say, Saudi Arabia and Oman is spent promoting UK equipment. The situation will be different for different countries, but an average of a token 1% is used to represent the role of embassies. FCO diplomatic spending plans for 2003-4 are £869m. Hence, CAAT allocates a subsidy of £9m.'

Do you see all the glaring, ridiculous problems with how they arrived at their guess of 24 mil? They’re basically just making things up. I mean, I don’t even know how Owen could look at this and think it was in any way reliable. HOW CAN THIS BE HIS SOURCE?! It boggles the mind.

Okay, now that “£670 million on research and development.” Another guesstimate. Here’s how they explain it:

'That there is a research & development (R&D) subsidy is acknowledged by the Government in that it charges a Commercial Exploitation Levy in an attempt to recoup some of the costs. The CEL brought in £17.5m in 1999-2000 (the most recent figure available, though the CEL was greater than this in most previous years), less than 0.5% of the value of military exports. Attempts by government over past decades to implement a more realistic levy (up to 30% of the selling price) have been watered down to 'what the market will bear', or what the companies will pay. In order to provide an estimate of R&D subsidy, CAAT simply applies the proportion of UK military equipment that is exported, approximately a third, to R&D spending. In 2001-2, R&D spending was £2,057million and so the amount that could be allocated to exports is £686m. This, minus the CEL of £17.5m gives a subsidy of around £670m.'

So that "£890 million each year" in government subsidy to the arms export business? It's all cobbled together from 10-year-old data or older, shaky estimates and numbers that are just pulled from the sky, basically. And it includes a government organisation that hasn't existed for seven years.

If you read the whole factsheet, which I did and I assume Owen did, it’s very clear that this information can’t be presented in any way as being accurate or up-to-date. And yet that’s exactly what Owen Jones does, presents it as accurate and up-to-date. Is he playing games or is he really just incapable of understanding how proper research is supposed to be conducted? This isn't from his personal blog where one could say, "who cares, what's the difference?" it's in a Sunday Times bestseller published by Penguin and being promoted all over the country and on television.

If you read this paragraph he wrote in his book and didn’t bother to check his source, you would believe that these were accurate figures that represented the current situation. But they’re all figures from a decade ago and older and much of them have just been made up by this group that he’s citing, all of which is plainly obvious when you read the source. I just don't understand how this could fly.'

Henry also asked me to emphasise that he isn't taking on Jones' argument that the arms industry is subsidised by the state. 'I'm just pointing out that his research is awful. And that the numbers just can't be verified the way he's claiming they are.'