Research on Open Data and Transparency

Leave a comment

Open Government in Australia and the UK


Birkbeck hosted a discussion comparing the openness of Australia and the UK, looking at the Australian government’s OGP commitments. The Open Government partnership is an international ‘multilateral initiative that aims to secure concrete commitments from governments to promote transparency, empower citizens, fight corruption, and harness new technologies to strengthen governance’. Governments who take part sign up to a set of commitments within a certain time frame and are then publicly assessed on how they implement them.  More than 70 countries have been signed up, as well as 15 sub-national governments.

Daniel is the assessor of the Australian government’s openness commitments, and interviewed NGOs and officials to write a report on how far the commitments have been implemented. Australia has a longer history of openness than the UK, having passed an FOI law long ago in 1982. This is not to say all is well and, as Daniel pointed out, secrecy has surrounded many of Australia’s activities not least its Trump-esque refugee ‘turnback’ policy and horrific stories from its offshore detention centres.

Australia was invited to join the OGP in 2011 but took a long circuitous journey to get there, as Daniel explains. It is currently on its first National Action Plan, with its Second National Action Plan 2018-20  due by the end of August, 2018 (the UK has just finished its Third and will soon be on its Fourth). Daniel’s first report was published for public comment in April.

As Daniel explained, there were many similarities with the UK’s own policies. Australia’s plan covers similar themes to the UK, highlighting integrity and private sector openness. Like Britain, it is pushing a Beneficial Ownership register opening up who has control of businesses, as well as extractives openness (a very big issue in resource-rich Australia) while also opening up data and ‘re-booting’ existing provision around FOI or elections. You can see how it overlaps with what the UK has been doing here.

Not everything has been smooth and there has been some resistance and foot dragging along the way. One key issue, as seen in other countries, is around the extent to which civil society, who must co-create the plan, is involved. A survey of members of Australia’s civil society network in early 2018 found that there were ‘hopes and disappointment’ with members expressing their ‘disappointment with the limited progress made on some commitments and the failure of most lead agencies to engage with civil society in a way that reflects the true spirit of partnership’. As happened with the UK earlier on, commitments have been driven from the centre with less input from either civil society or other levels of government (state or local), where interesting openness experiments often take place.

Some of the patterns in Australia are not new. There are cycles of enthusiasm and interest and governments go on and ‘off’ openness (more often off). There are also different levels of engagement between departments and often a slow down once commitments are made. This is also where CSOs come in as a force for pressure, and to build relationships.

As with openness more generally, leadership is key. Senior politicians need to be involved and enthusiastic to provide momentum. So far, there has silence from large parts of the Australian government.

You can hear the podcast here

You can see Daniel’s report here and a summary here.

Daniel is a senior lecturer at the ANU College of Law. Daniel is the independent Research Monitor for Australia as part of the international Open Government Partnership, reporting on developments relating to access to information in Australian Commonwealth, State and Territory Governments.





Leave a comment

The Scottish Information Commissioner’s Intervention: skulduggery in Scotland?

Today the Scottish Information Commissioner published his report into claims that the Scottish government had been making certain FOI requesters (namely journalists and political researchers) subject to different procedures. This followed claims from Scottish journalists in 2017 of interference in their FOI requests. Here’s a few conclusions from the report…

Were journalists dealt with differently?

The SIC concludes they were.

‘While I received reassurances throughout my interviews that journalists’ requests were dealt with in the same way as requests from any other person, this is clearly not the case.

Journalists, together with MSPs and political researchers, are expressly made subject to a different process for clearance than other requester groups. As set out above, their requests are almost invariably subjected to an additional layer of clearance which is likely to delay the consideration of the case. This process is applied because of who/what they are, not what they asked for. This is far from the applicant-blind principle of freedom of information legislation’.p.24

‘It may very well be the case that many requests for information from journalists, MSPs and political researchers are for sensitive information, in which case it may be entirely justified that clearance is required at a higher level in the organisation. However, by creating and applying a process based on requester type rather than the nature of the request, not only is the spirit of FOI legislation offended, but trust between those groups mentioned in the policy and the Scottish Government may also be damaged. I have heard criticisms of a two-tier system, and the existing policy simply reinforces such concerns’. P.24

Did it affect disclosure?

There’s no evidence that the ‘two-tier’ approach changed disclosure.

‘Except for 2015/16, the statistics do not show journalists to be treated in a materially different way from other requester types, insofar as the likelihood of obtaining full or partial disclosure is concerned. However, given the level of involvement that special advisers have in the handling of many information requests, there is obviously a perception that their involvement is disadvantageous to such requests’.p.27

…But  it did delay journalistic FOIs, above the average amount of time taken…

‘Despite these significant improvements, there is still a noticeable difference in time taken to deal with media, as opposed to non-media, requests. While some of this may be due to the complexity of some of those requests, it is inevitable that higher levels of clearance will add time to any response process’. P.30

Was any there wrongdoing (Part 1 identity)?

The SIC concludes it contravened the ‘spirit’ of FOI

‘In my view, the practice of referring all media requests for clearance is contrary to the spirit of FOI legislation. In most cases, the identity of a requester should be irrelevant for the purposes of FOISA and an authority should handle requests on the basis that they are applicant, and purpose, blind’ P.30.

Was there wrongdoing (Part 2 use of exemptions)?

It seems not-though exemptions were ‘pushed’ to their limit or tried out.

‘I could find no evidence of improper motives in the application of exemptions… There was an indication in some cases of reliance on exemptions where, although there may have been a legally stateable basis for doing so, the prospects of success, were the case to be reviewed or appealed, were not high’…though he notes one ‘flimsy’ attempt. P.31

Was there wrongdoing (Part 3 records and a chilling)?

One recurrent theme of the report was that there was a lack of documents to either support or refute claims (and the IC recommends there be new rules on record keeping).

‘The clarity of the situation is not helped by the general lack of records of interactions between special advisers and case-handlers in the case files… The lack of a clear record of what was discussed only feeds speculation which a clear record could dispel.’ P.23

‘Looking at the advice provided to Ministers, as previously mentioned, records management in case files is sporadic and in many cases the rationale for the decision is not clear from the documents.’ P.32

‘The examination of Scottish Government case files revealed significant gaps in the information recorded. In many cases, there was scant information contained in case files; in some there was no documentation whatsoever…Consequently, in many cases examined by my officers it was impossible to ascertain what processes had been followed, what (if any) discussions had taken place, whether advice had been sought and/or received and who had been involved in shaping responses’.p.34

Leave a comment

Fight, Hide, Avoid: Resistance to Freedom of Information Laws

trump-oval-office-advisorsAlthough there is no clear evidence of any government-wide, systematic resistance to freedom of information laws in the UK or elsewhere, there are examples of avoidance and resistance, often at very high levels and in sensitive situations. There appear to be numerous strategies to avoid or mitigate FOIs. Some clearly contravene the law while others run against the spirit of openness. Examples include:

Use of private email systems. In the UK, Michael Gove was found to be using a private system when he was Education Secretary, leading to an ICO ruling in 2012 clarifying the distinction between public and private communications (see below). Allegations have been made about other politicians trying to circumvent FOI through private emails, from Sarah Palin to various members of Donald Trump’s team, including, according to CNN, Steve Bannon, Jared Kushner and Ivanka Trump. Most famously, Hilary Clinton’s use of a private server and email may have violated the Federal Records Act and later archiving regulations, though whether she did this simply for convenience of access (as Clinton claimed) or to hide from FOI requests and potential Congressional investigation (as her opponents alleged) is unclear.

Non recording of information (known as the ‘Chilling Effect’). This type of non-minuted, informal mode of decision-making can take many forms, stretching from using phone calls, verbal decisions or disposable post-it-notes to having informal meetings entirely without records. The Swedish government, with the world’s oldest openness law, calls this the ‘empty archives’ phenomenon and mentions of ‘post-it-note’ approaches date back to the US in the 1960s and Australia in the 1980s. There was some recent evidence of non-recording in Scotland when, in November 2017, the Ferret unearthed evidence that the Scottish Environment Protection Agency asked for ‘verbal’ updates instead of written documents, mentioning FOI as a reason (see the emails here on page 11-12). This chain from 2008 shows a UK official apparently urging others to delete emails because of FOI.

Informal meetings, made famous by Tony Blair’s (pre-FOI) ‘sofa government’ style, are more difficult to prove. However, in March 2018, giving evidence to the RHI ‘Cash for Ash’ Inquiry, the Head of Northern Ireland’s Civil Service admitted that ‘the practice of taking minutes had “lapsed” after devolution’. The BBC quoted him as saying “Ministers liked to have a safe space where they could think the unthinkable and not necessarily have it all recorded” and the ‘DUP and Sinn Féin were sensitive to criticism and in that context, senior civil servants had “got into the habit” of not recording all meetings’. He said this was done on the basis that it was sometimes “safer” not to have a record which might be released under Freedom of Information.’ He did, however, qualify this and ‘agreed with the inquiry panel that when it came to ministerial decisions on matters of public money it should be recorded.’ Former Head of the UK Civil Service Gus O’Donnell had made similar claims for the UK Cabinet being ‘hamstrung’ by FOI, though relied on anecdote and hypotheticals to illustrate his point (see this analysis).

Involvement of press advisors and media officials. The Scottish Information Commissioner spoke of claims of ‘deliberate delaying tactics and requests being blocked or refused for tenuous reasons’. One study of more than 2000 ATIA requests in Canada by Professor Alasdair Roberts found exactly this happening, with ‘requests that were identified as sensitive, or which come from the media or political parties, found to have longer processing time, even after other considerations are accounted for’. Procedures within government focused on ‘giving special attention to politically sensitive requests for information’, ‘flagging’ difficult requests electronically and using communication officers and Ministerial offices as gatekeepers for releases. Similar claims were made in the UK for the ‘Clearing house’ that was intended to co-ordinate responses across government.

Timing, requesting and release. One approach seen in Ireland and elsewhere is for government departments to deprive journalists or others of scoops by publishing request responses online simultaneously. Alternately, releases can be buried by publication on a busy news day or at the end of a week. Even more creative has been the approach of the FBI, who recently asked that FOIA requests only be sent by fax (though the FBI FOI email appears to still exist).

One important point to make is that while high profile examples exist, mainly from politicians, we found officials were generally as fearful of not having a record as having one, with the pressure from bosses or potential judicial review to keep a record outweighing the danger from FOI. Research at local government level in Scotland and the UK found there to be a positive, professionalising effect on records.

Two problems: Is it happening? Is it FOI?

Any research in this area hits two major difficulties.

How do we know it is happening? It is unclear how widespread such activity is. Are the examples above are the tip of the iceberg or isolated misbehaviour? By its nature, it is very difficult to prove a negative (i.e. something didn’t happen) in terms of records or avoidance and is often reliant on anecdote. One study found many comments to ‘not record’ were often jokes or light hearted quips. However, the problem is that the myth could become reality. One previous UK Information Commissioner warned that constantly ‘talking about a chilling effect’ can make it a ‘self-fulfilling prophecy’, so when Tony Blair claims FOI laws make politicians ‘very cautious’ or David Cameron argues it is ‘furring up the arteries of government’ they could make it happen.

How do we know it is FOI? Isolating and disentangling FOI as the cause of the problem is almost impossible. Fear of leaks, the arrival of new technology, new decision-making styles and the key question of resources all influence how and if records are kept. Ex-Downing Street Communications Director Craig Oliver complained that accusations made during the Brexit referendum campaign that Cameron was avoiding FOI by using WhatsApp misunderstood that it was simply quicker and more secure than email. Concerns over the non-recording of information go back far into the past. As the Justice committee pointed out, the 2004 Butler report raised serious concerns over Tony Blair’s use informal meetings and ‘sofa government’ a year before FOI came into force. James Callaghan’s famous quip captures the nuances and grey area between official and unofficial disclosure: ‘I brief, you leak’.

What can be done?

There exists rules on records and destruction in almost all FOI laws (and Scottish FOISA has them in section 60 and 61). In March 2018 the first charges were bought for a Councillor in Thanet ‘blocking, concealing or destroying records held by Thanet council “with the intention of preventing the disclosure by TDC of information”. This also touches on rules around whether emails are archived or deleted and more generally what records are kept and for how long, issues that proved controversial recently in relation to Scottish care homes and a panel looking into paramilitary disbandment in Northern Ireland.

Some laws also cover private emails being used for public business. In 2012 the UK ICO, for example, created new guidance in wake of Gove email controversy, making clear that texts or emails are covered if ‘information “amounts to” public authority business’ or “generated in the course of conducting the business of the public authority”.

In 2017, former Scottish Information Commissioner Rosemary Agnew called for a ‘radical re-think’ around how FOI works in terms of disclosure, recording keeping and release, so bodies ‘invest in dissemination of information as a way of doing business’ rather than through requests.

The letter from Scottish journalists in 2017 called for a specific legal duty to record decisions. The UK information Commissioner has also championed the creation of a ‘duty to create records or “duty to document”…a positive duty in law to create records of significant decisions, actions and events’. That means public bodies must keep ‘records explaining and providing context to why a specific course of action was taken [including] Minutes of important meetings, decisions, that led to policy change and new initiatives’. Such a law exists in British Columbia following an email deletion scandal, though campaigners claim the law lacked teeth as it did not mandate the creation of records.

Further Reading

  • Worthy, Ben and Hazell, Robert, Disruptive, Dynamic and Democratic? Ten Years of Freedom of Information in the UK (December 28, 2015). Parliamentary Affairs, Forthcoming. Available at SSRN:
  • Roberts, Alasdair S., Spin Control and Freedom of Information: Lessons for the United Kingdom from Canada (December 20, 2003). Public Administration, Vol. 83, No. 1, pp. 1-25, Spring 2005. Available at SSRN:




Leave a comment

Freedom of Information and Universities UK: Could it? Should it? Why Should It Worry UKIP?


Could Universities UK be covered by the FOI Act, as this e-petition asks? I’ll try to answer this, using the old EU joke, to see if it works in theory as well as in practice.

In the UK, universities are covered by FOI (as Vice Chancellors have found out) but UUK is, as of this moment, not. You can’t really ask a body not covered (though this hasn’t stopped people trying). So could and should UUK be covered?

Doubt it…

Most FOI Acts cover ‘public bodies’ and, to very different degrees, private bodies doing work for them. This means legally ‘bodies that appear to be exercising functions of a public nature or who are providing, under contract with a public authority, any service whose provision is a function of that authority.’ Only two Access to Information Acts in the world directly cover private bodies as a whole (South Africa and Nigeria seeing as you asked/need to know in case of some unimaginably obscure pub quiz).

The legal status of UUK looks pretty far from being a public authority. UUK is the ‘voice’ of UK universities made up of the ‘vice-chancellors or principals (executive heads) of universities in the UK’. Legally speaking it is a ‘Company Limited by Guarantee’. This means, according to my brief foray on Wikipedia, it is like a private company but, as pointed out here, ‘a major difference is that it does not have a share capital or any shareholders, but members who control it’.

This status is often used for charities and others to prevent liabilities. According to Wikipedia[1], common types of companies limited by guarantee are:

So, a big fat no in theory?

On a practical level, it takes ages. Gordon Brown (that clever PM who didn’t call snap elections) took two years just to think about who to extend it to. By the time he’d decided and extension was begun, David Cameron (the one who called snap referendums) was in power. A full five years.

Or Could It?

The government can extend the FOI to cover broadly who they want. Section 5 of the Act allows government to re-designate bodies (sorry to get a bit legal), roughly as long as it has a ‘number of definable relationships with public authorities’.

The first actual section 5 extension (in November 2011) designated the Universities and Colleges Admissions Service (UCAS), the Association of Chief Police Officers (ACPO) now known as the National Police Chiefs’ Council (NPCC) and the Financial Ombudsman Service (FOS).

As of May 2018, you’ll be pleased to know, another 15 bodies will be covered including my favourite Independent Monitoring Board The Independent Monitoring Board for the Military Corrective Training Centre and my second favourite development Corporation the Ebbsfleet Development Corporation. The separate Scottish FOI Act has been extended to cover ‘secure accommodation for children and young people, grant-aided schools, independent special schools and Scottish Health Innovations Limited’.

But even if the government want to do it, can they justify it or, to quote George Galloway (rather out of context), are they ‘mad enough to do it?’ A broad moral case to open them up could be made. Many other university bodies in the same ‘regulatory ecosystem’, admittedly of very different legal status, such as UCAS or HEFCE are covered by FOI, as is the Student Loans Company (go on-what are you waiting for?). As the ‘voice of universities’ it also does seem to at least appear to be exercising functions of a public nature’ and has a ‘number of definable relationships with public authorities’. The Public Accounts Committee, among others, has also repeatedly called for FOI to ‘follow public money’ and post-Grenfell there was a further attempt by Private Members’’ Bill to extend FOI.

Even the fact it is a ‘Company Limited by Guarantee’ may not totally exclude it. What’s interesting is that almost all the wikipedia list of examples have been the subject of discussion about extension. In 2009 and again in 2016 (post Kids Company) there was some serious discussion of making charities subject to FOI. Though various investigations have concluded they shouldn’t be, the possibility is still there. Even more fun, Network Rail came to be covered by FOI almost by mistake, when its legal status changed for accounting purposes (you can see what people have been asking here-I now know how many platforms every station in the UK has and the length of all of Reading station’s platforms).

Political parties aren’t covered by FOI in the UK but in some countries, such as Indonesia according to an ongoing discussion, it seems they are. There was a hellish row in India about trying to get Indian political parties covered by their Right to information Act and, seeing as we are talking cricket, pressure to make the Indian Board of Control for Cricket a public body for the purposes of the law.

And so?

Technically, UUK is not covered. For the present, you can ask for ‘material held by a private company “on behalf of” a public authority’. This is very confusing and no one really knows what’s going on but it may mean some UUK material is covered. Moreover, many of the public authorities UUK interacts with, meets and emails-such as all the universities where all the VCs are based, the government departments they work with etc-are covered by FOI.

Even if Theresa May decided, in yet another flush of pointless and futile Brexit distracting incompetent zeal, to get out of the chimney she was in and extend the law (thus robbing yet another Ed Miliband policy), it would take time. One thing to reflect on. The more such bodies are covered, the closer we could get to getting UKIP covered by FOI. Oh, the emails, as America’s new racist in chief would say.

See my piece on VC expenses here and on opening up the private sector. There’s some better legal analysis by FOI man here and a post on wider legal openness developments from Act Now.

[1] Note to my students: don’t ever use or cite Wikipedia for essays. I am doing it ironically.

Leave a comment

My FOI: A list of all the FOI requests rejected as ‘vexatious’ since 2016


DEXEU hasn’t covered itself in glory in FOI terms, as the IFG have demonstrated. So I asked what requests had been labelled vexatious.

My FOI, made via WhatDoTheyKnow, asked

‘Dear Department for Exiting the European Union,

This is an FOI request for a list of all the FOI requests (in particular the detailed question or questions in each request) rejected as ‘vexatious’ since 2016.

Yours faithfully,

Ben Worthy’

Here’s the response:


Reference and Wording of request


* Please provide a breakdown of the £533 cost of a trip by David Davis to Belfast from Aug 31 to Sep 1, 2016. In providing the breakdown, please disclose the cost of the flights, the cost of accommodation and the cost of all meals. Please also provide copies of all receipts, bills and invoices relating to the trip.

* Please provide a breakdown of the £273 cost of a trip by David Davis to Dublin from Sep 8 to Sep 9,Please also provide copies of all receipts, bills and invoices relating to the trip.

* Please provide copies of all receipts relating to subsistence and meal claims by Oliver Robbins in relation to: * a trip to Paris/Rome starting on August 23; * a trip to Warsaw/Budapest starting on August 3;

* a trip to Brussels on September 5;

* a trip to Brussels starting on September 21;

* Please disclose the cost of the business premier rail travel relating to trips by Oliver Robbins starting on July 19, September 5 and September 21.


I would like all correspondence of decisions on the attendance of Ministers and Secretaries of State for European Union Exit and Trade Cabinet Committee meetings since July 2016.


I would like details of any minutes, agendas or briefings sent by the Cabinet Office to the Secretary of State for Wales, regarding the European Union Exit and Trade Cabinet Committee since July 2016.


Under the freedom of information act, I would like all correspondence of decisions on the attendance of Ministers and Secretaries of State for European Union Exit and Trade (Negotiations) Cabinet Sub-Committee meetings since

July 2016.


Under the freedom of information act, I would like all correspondence of decisions on the attendance of Ministers and Secretaries of State for European Union Exit and Trade (International Trade) Cabinet Sub-Committee meetings since July 2016.


Under the freedom of information act, I would like all correspondence of decisions on the attendance of Ministers and Secretaries of State for European Union Exit and Trade (European Affairs) Cabinet sub-Committee meetings since July 2016.


I would like details of any minutes, agendas or briefings sent by the Cabinet Office to the Secretary of State for Wales, regarding the European Union Exit and Trade (Negotiations) Cabinet Sub-Committee since July 2016.


I would like details of any minutes, agendas or briefings sent by the Cabinet Office to the Secretary of State for Wales, regarding the European Union Exit and Trade (International Trade) Cabinet Sub-Committee since July 2016.


I would like details of any minutes, agendas or briefings sent by the Cabinet Office to the Secretary of State for Wales, regarding the European Union Exit and Trade (European Affairs) Cabinet sub-Committee since July 2016.


Under the freedom of information act, I would like details of any minutes, agendas or briefings sent by the Cabinet Office to the Secretary of State for Northern Ireland, regarding the European Union Exit and Trade Committee since July 2016.


Under the freedom of information act, I would like details of any minutes, agendas or briefings sent by the Cabinet Office to the Secretary of State for Northern Ireland, regarding the European Union Exit and Trade (Negotiations) Cabinet Sub-Committee since July 2016.


Under the freedom of information act, I would like details of any minutes, agendas or briefings sent by the Cabinet Office to the Secretary of State for Northern Ireland, regarding the European Union Exit and Trade (International Trade) Cabinet Sub-Committee since July 2016.


Under the freedom of information act, I would like details of any minutes, agendas or briefings sent by the Cabinet Office to the Secretary of State for Northern Ireland, regarding the European Union Exit and Trade (European Affairs) Cabinet sub-Committee since July 2016.


Leave a comment

How Open is Britain in 2017 and Where Next?


To celebrate International Right To Know Day, the Centre for British Politics and Public Life held a panel discussion on how Open Britain was. The UK has seen more than a decade of continuous openness reform, from Freedom of Information and Open Data and all sorts of information on gender pay gaps and experiments with election data. But where are we now?

Our panel of experts, Martin Rosenbaum (Journalist, BBC), Rosemary Agnew (Former Scottish Information Commissioner and now Scottish Public Services Ombudsman) and Professor Sarah Childs (Professor of Politics and Gender, Birkbeck College) debated how open the UK really is in 2017 and where we could go next.

If you want to know about the openness of Britain’s political candidates and restaurant hygiene, why FOI is not always enough and how Brexit could take us backwards, listen to the podcast below.

Listen on Sound Cloud

For further reading