Research on Open Data and Transparency

Leave a comment

Extending FOI?


Should the Freedom of Information Act (FOI) be extended to include private bodies doing public work? In the past the Public Accounts Committee (PAC) has argued it should in a report on PFI projects. In July 2012 it recommended that Network Rail, currently outside of FOI, should be covered. Nor are they alone. TheLocal Public Data Panel, a group of experts overseeing transparency and Open Data reformed across local government, warned of an erosion of public access due to the contracting out of services.

Why do it?

Extending coverage to private companies running public services, such as rail or road providers, is on the face of it logical and popular. The Coalition Agreement states it as an aim and Labour have committed to it since 2012. However, the devil lies in the detail and in the persuading of companies to do it, particularly now. But, while politicians prevaricate, some information is creeping out by the back door.

Most laws cover only information held by authorities, which can include some information held about work done by private bodies. Many companies are happy to provide information but not all, as our research on FOI and local government shows.

In the UK, Gordon Brown was the first to suggest extension of FOI to private providers in 2007. After several years of consideration it was decided to cautiously extend it to only a limited number of bodies. In 2010, the issue of extension surfaced again when new Deputy Prime Minister Nick Clegg aired the possibility that FOI could cover a wide variety of new bodies from Network Rail to utility companies. The Conservatives had also pressured the previous government to cover the nationalised bank Northern Rock.

Meanwhile, over the border in Scotland, the Scottish government began to consult on its own extension to selection of private bodies under its separate FOI Act. Again, following a long process, the final decision was to not do so.

Why Not?

Why have so few governments tried to extend coverage? As the UK and Scottish experience shows, even discussing extension can be a time consuming business, requiring a great deal of consultation and consideration.

More importantly, the arguments against extending FOI can be persuasive ones. Businesses argue that they can’t afford it (‘it’s too expensive’) and don’t need to (‘it’s unnecessary as we publish most of this already’). Here is the list of reasons for not extending FOI cited by the Scottish government, which would probably be enough to give even the most pro-openness politician pause for thought:

  •  ‘No compelling evidence of a problem or of unmet demand for information.
  • Considerable information is already available through the relevant public authority – as well as by various statutory and regulatory means.
  • Wide concern about the potential resource implications and administrative burden of extension – particularly in the current economic climate
  • Most contracts stipulate co-operation between contractor and authority regarding information requests. Extension to contractors could be deemed to be a ‘discriminatory change in law’ – with costs passed to the public sector
  • Potential issues arising from differing regimes operating within the UK – including competitive disadvantage
  • Concerns that coverage would impact on private business engaging with the public sector and that resulting costs would be passed on to public authorities
  • That extension would be contrary to Scottish Government’s aim of reducing unnecessary regulation’

Creep in By the Back Door?

 Interestingly, other developments and innovations may mean some level of transparency, through FOI and now Open Data reforms, may slowly ‘creep up’ on private companies. In Ireland, for example, it was recently decided that the Ango-Irish bank could be subject to Environmental Information Regulation requests (an equivalent FOI for environmental matters).

In the UK the Ministry of Defence has begun ‘naming and shaming’ apparently poorly performing contractors. At local government level, one high profile joint venture between IBM and a set of public bodies has explicitly committed itself to FOI and many authorities are determined to make FOI access part of future contracts. In parallel, sites such as Openly Local or Contracts Finder allow the public to find out more about, for example, councils and their suppliers. The Justice Committee’s 2012 scrutiny of FOI concluded that openness clauses in contracts were the best way to move forward (see from para 233 onwards here).

Use of FOI may even creep outwards, as it has here with Train Companies asked for salary figures (train companies are not covered by FOI but one answered a FOI request anyway). Only one FOI law in the world currently wholly covers private bodies, South Africa’s Promotion of Access to Information Act 2000, though it’s not clear if or how it works. The Indian Right to Information Act 2005 also extends to previously public utility bodies now in private hands, though this was due to a ruling by its appeal body rather than by government action.

This ‘creep’ is no replacement for full FOI coverage but the whole issue of extension leaves politicians in a dilemma. They wish to hand the public more power to hold public services to account but are reluctant to upset business or add any regulatory burden, especially now. Will the onward move of transparency and Open Data leave them with no choice?

The first version of this article was originally published on the LSE policy blog.

Leave a comment

What is Transparency?


Roughly we can agree, as Professor Christopher Hood and Professor David Heald put it, that transparency means letting ‘those on the outside being able to look in’ (‘peering through the window’-like the glass of the German Reichstag above).

But if we dig deeper it gets complicated. Not everyone agrees on what sort of transparency they want. In fact, how you define it helps you decide whether it is good or bad.

Take Freedom of Information. On one level the law itself is quite clear. FOI is the right to ask questions of a public body, bounded by rules. The problem is not everyone sees eye to eye on what ‘real’ FOI means and how it is working.  David Cameron said ‘real freedom of Information…is the money that goes in and the results that come out’ and claimed that requests about processes or decision-making were ‘furring up the arteries’ of government (see FOI man’s analysis here). In David Cameron’s mind it is more about driving economic recovery than visitors to Chequers.

Tony Blair went further and spoke about how FOI, intended for ‘the public’, was being abused by opponents

The truth is that the FOI Act isn’t used, for the most part, by ‘the people’. It’s used by journalists. For political leaders, it’s like saying to someone who is hitting you over the head with a stick, ‘Hey, try this instead’, and handing them a mallet (Blair 2010, 516-517).

Passing FOI was one of his two biggest regrets. Obama is fast becoming another leader who went from first term transparency advocate to second term holder of secrets. It is unlikely that his offer of NSA transparency via a website to help inform the public about the intelligence service will satisfy his critics.

There is similar confusion around Open Data. It’s not clear whether Open Data is about letting people access information, a sort of ‘FOI plus’, or about re-using it. As this paper points out, Open Data mixes all sorts of political and technical ideas, hopes and concepts that don’t fit very well together.

It’s still not clear exactly what the UK government wants to see happening with all this data. Is all this new online data designed to make the economy grow? Or is it there to get the public more involved in politics or better understand what government does? Nor is it clear what information the public want: is it about salaries or road salting? (see my discussion of this in evidence to the Public Administration Committee here). There are signs, as Jonathan Gray recently pointed out, that the UK government is seeking to shift the aims of its Transparency Agenda, changing the narrative from the ‘political’ to the ‘economic’.

The difficulty is that transparency is many things at once. It is partly economic. It is also about making politicians accountable for what they do-sometimes they are minor matters and sometimes big ones (and odd ones such as MPs’ portraits). It is also there to help NGOs and others, providing a new weapon in their armoury to campaign against everything from library closures to polluted air. Yet it can be, and often is, a practical tool to help people in their everyday lives. For all the attention given to MPs’ expenses, FOI or online data is most often used to help individuals.

So how can we define it? George Orwell defined liberty as the right to tell people what they don’t want to hear:

‘If liberty means anything at all it means the right to tell people what they do not want to hear’ (from his unpublished preface to Animal Farm)

We could try and rework this to fit transparency:

‘Transparency is the right to ask questions those in power don’t want asked and look for information they don’t want us to see’.

This not to say this is how it used. In fact the evidence is most people use Open Data or FOI in a relatively non-political way. But this is the key, driving idea. The problem for the future is that, without agreement on what it is, we can’t agree on what transparency policies can do and who they will benefit. It is even harder to see if the policies are a success or not when we don’t agree what success means.


Where are the Armchair Auditors?

armchair by Didouner One of the most eye-catching ideas from the government’s Transparency Agenda is that of the ‘Armchair Auditor’. Armchair Audit is a form of ‘crowd sourced’ accountability where citizens use newly ‘opened up’ information on spending, contracts and crime to hold public bodies to account. But has it happened?

David Cameron hoped a critical mass of citizen watchdogs would become a new force for accountability.As he put it a ‘whole army of effective armchair auditors looking over the books’ would act as a check on ‘waste’. Citizens being ‘able to see how your taxes are being spent’, ‘judge standards in your local schools and hospitals’ and ‘find out just how effective the police are at fighting crime’ would drive better performance and efficiency.

Much of the emphasis has been financial and local due to the publication of local authority spending over £500 (see this LGA survey for background). Eric Pickles put it bluntly-the data would ‘unleash an army of armchair auditors and quite rightly make those charged with doling out the pennies stop and think twice about whether they are getting value for money’.

Yet the army has not appeared, as the BBC pointed out. This is not to say there are no auditors. Adrian Short has created an open access software taken up by Auditors in the‘North’ (Hull and Lincolnshire) and on the Isle of Wight. In London one particularly high profile case led to Eric Pickles publicly praising Armchair Auditors in their fight against Barnet’s ‘easycouncil’-see this account here. Journalists offered tips and ran storiesand the Taxpayers Alliance promoted it as a grassroots weapon.

But this vanguard shows little sign of becoming a citizen audit army. There are a number of reasons for this, some obvious, some less so.

The first and most obvious barrier is the information. It is not yet consistent so questioning and understanding it is not easy or possible. The Reluctant Armchair Auditor and the Public Accounts Committee have expressed concern at the quality and comparability of information. This may improve in the future, as this FOI response from DCLG suggests, with new applications making it easier to compare.

The second obvious point is that to be an Armchair Auditor you need to be a particular type of person. Politically, you need to be engaged and interested in local government, understand how local government works and have a driving reason to dedicate yourself to it. To have all these traits in combination is rare-the public are, by most measures less engaged, though there may be glimmers of hope for local government. Add to this the time and the need to access technological know-how and this narrows down the potential pool of Armchair Auditors to a select few. Even then there is no guarantee as this would- be-but-frustrated Auditor explains.

The third barrier is what you do with the information. Like FOI, it is hoped that information equals accountability. But there is a missing link here-what do you do with the information once you have it? Do you pass it to the authority itself? A member of the opposition? The press? Or do you self-publish and hope it is picked up? As this discussion on the E-democracy blog shows, it is not clear exactly where this new information fits in. Again like FOI, the data is likely to work alongside other accountability tools-the activists in Barnet have used a combination of FOI requests, the local media and judicial review.

A final point is that Armchair Auditors, like all accountability mechanisms, work best in certain situations. A high profile controversy or scandal, as in Barnet, appears to be one way of kick starting them. A lack of activity from the official opposition or local press may drive groups to do it themselves- so auditors can fill accountability vacuums. The paradox is that for Auditors to then be effective they often need to use these other channels that failed.

David Cameron pointed to the MPs’ expenses scandal as being an example of what could happen when information was released. Yet this scandal actually illustrates how complex accountability can be. It was driven not by citizens but by determined journalists pushing a case through appeal and the courts. The key moment came via a good old fashioned (paid for) leak. It took four years, persistence and a whole range of accountability tools working in a very particular context.

As this paper argues, crowd sourcing exercises are often fragile and subject to bias, relying on a ‘tiny subset’. Auditors, like many other such initiatives, are better seen as a compliment to existing systems than a replacement. Auditors will rise up in certain situations and the new information will form part of the armoury of activists, journalists and NGOs. However, the citizen army is likely to remain in the barracks.

This post was originally published on the ODI blog

Leave a comment

Publishing Local Government Spending Online: Was It Worth It?

The UK government is committed to leading the world in openness. As part of the push to transparency, all English local councils have been asked to publish all their spending over £500 online on a monthly basis. These ‘swift and simple changes’ could ‘revolutionise local government’ and ‘unleash an army of armchair auditors’, with citizens analysing the data and holding their local councils to account.

For the last year I’ve been researching the impact of the publication, using a combination of surveys, FOI requests and interviews. I’m asking whether making this new financial information available has had an impact: are members of the public using the new information to make local government more accountable? Do they to participate more in local politics? Do they understand more about their local council?

The evidence to date shows a limited number of people are looking at the information. A mixture of journalists, local activists and a few citizens are using the data to hold councils to account and there have been some stories in the local press and the national press. There are a few armchair auditors appearing around the country such as here and here (especially, as Eric Pickles pointed out, in Barnet) but it takes time, resources and interest –something not many of us have.

There are obstacles. The data itself isn’t as informative as it could be, and it can’t yet be compared with other councils or give a full picture of what is being spent and how (see your own local council site-mine is here). Some councils are not convinced that the public will spend their time looking through spread sheets. When you contact your council it’s generally for amenities or services-bins or planning-rather than spending.

To make it more complicated the spending data is also very political: it isn’t clear exactly what the government wants to see and they are sending out ‘mixed messages’. Some feel the information, at a time of severe cuts, is intended to make councils look like reckless spenders or wasters.

The lack of use and obstacles does not mean the reform has failed. Far from it. Councils can see the benefits of all this information for both citizens and for themselves. It is very early days and the real changes will arrive when new applications such as Openly Local or (still in development) allow you to quickly search the data, compare spending with other councils and join it up with other ‘local’ information. In the future, you can imagine a combination of TheyWorkForYou or the Open Data Communities hub for your local council.

For more detail see my slides and lecture talk to the Open Data Institute here. You can see a conference paper here and some evidence given to the Public Administration Select Committee here

If you are interested or would like to help with my research please contact me on

This post was originally written for the Open Data Institute blog 

Leave a comment

How is new technology re-shaping politics?


This blog aims to bring together new research on Open Data and Transparency. It will look at the impact of Open Data and new Information and Communication Technology on government and politics. It will also follow the emerging theory and evidence in the UK and elsewhere, asking:

  • Does Open Data increase transparency and accountability?
  • What other benefits can it bring?
  • Does it have any unintended consequences?

Who I am

My name is Ben Worthy. I’m a lecturer in Politics at Birkbeck College, University of London. Since 2011 I have been studying the impact of Open Data and the new Transparency Agenda in the UK.

My latest work looks at the impact of the publication of all spending over £500 at local government level since 2010. You can see a conference paper here, some evidence given to the Public Administration Select Committee here and talk to the Open Data Institute here.

I have also written extensively on issues around Transparency and Freedom of Information. I was previously Research Associate in Freedom of Information at UCL’s Constitution Unit-you can see the results of the FOI team’s work here.


Staff web page

View my research on my SSRN Author page: