The material on Beyond Budgets is the responsibility of individual authors alone. It does not represent the views of any project, organisation or government.


What does an index of transparent finances tell us about accountable government?

Authors: Philipp Krause (Team Leader-Public Finance, ODI), Shakira Mustapha (Research Officer, ODI)

The International Budget Partnership recently published the 2015 Open Budget Survey and launched it at ODI last week. It sounds a bit underwhelming, but is actually a pretty big deal. The bi-annual survey is the go-to measure of fiscal transparency for researchers, investors and policymakers, covering a large number of countries since 2006. 

Fiscal transparency, broadly defined, is a right. In an open society, where citizens have a stake in what their government is doing and contribute to it by paying taxes, they should have a reasonably good idea of what governments are doing with public money.  Fiscal transparency can enable this.  

Unfortunately, the devil is in the details. Enumerating the different ways a government can be transparent about public funds and assigning weights and values to different elements is the greatest achievement of the indices of the Open Budget Survey.  At the same time this is the heart of the problem. There is always a question of how much they are a proxy for the actual issue of interest.  

The OBS’s usefulness as a measure of transparency is threatened by its own success. Any indicator can be, and as soon as it matters enough for policymakers, will be gamed.  Fiscal transparency has been shown to affect government’s borrowing cost and access to credit, and rating agencies use the Open Budget Index to assess transparency. Governments therefore have an incentive to try and score well on the index, focusing on the dimensions measured by the index and potentially neglecting other important dimensions.

The biggest problem for the survey, however, arises when putting it into context. It literally measures whether certain pieces of budget information are published in a certain way. It assumes that this is a specific and close enough image of what transparent public finances looks like. If viewed solely as an issue of fiscal transparency as a right, this is just fine. It is then important enough to know about transparency in its own right, irrespective of how it fits into the bigger governance picture (a government could have an open budget, but still steal from it).

Unfortunately, the OBS reports departs from this focus in two important ways, each of which is unhelpful to the cause of strengthening fiscal transparency – IBP is, after all, not just a disinterested observer, but an advocate for this cause.

Firstly, the OBS broadens its scope beyond transparency to what IBP calls the ‘accountability ecosystem’ in government. It complements the transparency index with two other indices, budget participation and legislative and audit oversight. As a response to the “fiscal transparency, so what?” question, the addition of these two dimensions is both too ambitious and too narrow. Although 24 out of 102 countries achieve fairly high transparency scores (above 60 out of 100 points), only 4 countries score more than 60 in all three dimensions. When the survey finds only 4 accountable governments in the world, it makes one wonder if perhaps this measurement isn’t quite right. And if the vast majority of countries scores very poorly, even countries that look pretty accountable otherwise, then maybe participation is not quite so important an element of accountability?   

Secondly, IBP argues explicitly that accountable government, as defined through the three pillars, will be critical for the implementation of the Sustainable Development Goals. Interestingly, the report mentions China nine times, elaborating that China runs one of the world’s least transparent, participatory or overseen public finance systems. It does not mention that China alone did the most of any countries in the world to make progress towards the MDGs. It is one thing to note that cross-country analyses suggest a relationship between transparency, accountability and development outcomes, quite another to claim that one is necessary to achieve the other. China is the most obvious example, but there are other country examples that reinforce this point.

There are two ways the OBS can filter into development practice. One is to focus on transparency on principle, even if that means that many governments won’t listen or won’t care, because they either feel no pressure or no inclination to share more. Pressure from domestic stakeholders and even external influences may change that calculus over time, and knowing where a government stands in the OBI will help. Another way is to try and sell transparency to governments, together with participation and oversight as part of an “accountability package” to help countries achieve development outcomes and implement international strategies. Donors might be tempted to use the results from the OBS, or specific indicators, as policy conditions for the disbursement of aid. It is far from clear how transparency, participation and oversight relate to development outcomes in any one country. To pretend otherwise is a development policy dead end.

PrintView Printer Friendly Version

EmailEmail Article to Friend

References (10)

References allow you to track sources for this article, as well as articles that were written in response to this article.

Reader Comments (6)

From Paolo de Renzio, Senior Research Fellow, International Budget Partnership

Dear Philipp and Shakira,

Thanks for your thoughtful comments on the 2015 Open Budget Survey. Let me provide some thoughts on some of the important points you raise:

1. All indicators, especially those of the governance kind, suffer from serious limitations. As you correctly say, often they are mere proxies for the real "thing" they try to measure, and they are prone to gaming. For the OBI, we chose recognized international standards as the basis for our indicator, and we formulated survey questions as objectively as possible. We do not claim to capture all aspects of what can be called budget or fiscal transparency, and other measures exist that complement our own and that people can resort too if they feel that the OBI is incomplete. Gaming the OBI through the questionnaire is not very easy for governments, as the research is carried out by independent researchers on factual information, and is verified by peer reviewers and the IBP. What can happen is that a government will publish additional information and/or additional documents to comply with the OBI and not to increase fiscal transparency as such. However, this will increase the de facto amount of budget information in the public domain regardless of the government's intention, bringing benefits to those who use such information. The fact that the research is done by local civil society groups means that in many cases there is a ready-made audience for the information. In cases where government commitment to transparency is lacking, the real issue is not gaming, but ensuring that changes in government practices are institutionalized, sustained and deepened.

2. The OBS looks at oversight institutions and at participation because we have seen that transparency by itself is not sufficient to bring about accountability. Official oversight institutions like parliaments and external audit agencies play an important role, but their weaknesses can and need to be complemented by more direct citizen engagement in budget processes. Moreover, all institutions can benefit from more interaction and collaboration. We do not claim that all of this is sufficient to guarantee accountability, and we are aware that these components of the survey are preliminary attempts in need of further refinement. We are already planning a thorough review to strengthen them. The message that very few countries achieve a good performance on the "pillars" of a good budget accountability ecosystem that we identify clearly deserves further attention and scrutiny, but it draws the attention in particular to the scarcity of participatory mechanisms that governments offer to their citizens. It is because of participation that only four countries pass our comprehensive test. Despite the gaps in the evidence, the fact that participation is important for accountability is in fact well supported by evolving international norms and by the evidence in the paper you provide the link to, rather than otherwise. Quoting directly from it, "The documented impacts of transparency are largely limited to intermediate steps – such as macro-fiscal outcomes or reduced corruption – in the long chain leading to development outcomes. To get further down the chain, the evidence seems to suggest that participation needs to kick in, to ensure that governments are better informed about and more responsive to citizens’ needs and priorities."

3. Your point about China is a well-rehearsed, but not necessarily a convincing one. China's contribution to halving global poverty levels is important, but not necessarily an example many are willing to follow. As far as I know, China is not a huge destination for people in search of a better life. And the SDGs recognize this by including a number of goals and indicators that try and capture aspects of well-being that go beyond socio-economic indicators (see for example goal 16 on "Peace, Justice and Strong Institutions"). It is quite likely that China will not shine on these aspects of development, including on issues of transparent, participatory and accountable budgeting. These will not by themselves deliver on the SDGs, but they are no doubt important components of a process that helps create the conditions for success, even if only by respecting people's right to know and be part of what their governments do with public resources.


September 29, 2015 | Registered CommenterRyan Flynn

Hi Paolo,

Many thanks for picking up some of our points in detail.

First off, I don't think we have any disagreement about the question of indicators and proxies. Of course they're never perfect, will be gamed, and more so the more is at stake, and so on. Every index involves value judgements and therefore reasonable people can always disagree about this part or that, and some countries will always be a better fit for some parts of it than others. None of this is a reason not to do it, it's just a health warning that says "handle with care". In the case of the OBS, it's certainly great to have and for good reason do researchers, policymakers and practitioners await the latest iteration with great interest.

I think we do differ a bit more fundamentally on the inclusion of the participation index, and I think the issue of transparency/accountability as a normative goal vs a means to something else plays into this in an important way.

If you care about transparency as a right in its own right, then that's where the discussion ought to be focused, intermingling it with other areas that may or may not be important in their own right just muddies the waters on that front.

If on the other hand you care about transparency because it is an important stepping stone to other things, then I seriously question your choices here. There are many things that complement transparency and could interplay with transparency to improve, say, accountability. Oversight, yes, and importantly oversight could be in service of a wide range of "principals". But the first things that would occurr to me are mechanisms like a free press, competitive elections and so forth.

So I'm just a little mystified how you get to participation. Especially since your own data seems to very clearly show that it isn't that terribly widespread (at least not in the way you measure it) and is not a necessary element for accountability or "development" to happen. To me, if there's such a stark difference in the distribution of the transparency and oversight scores on the one hand, and the participation score on the other, something's off with the "accountability package".

In this context, the findings of your paper are of course relevant, but a pretty narrow foundation. You're finding that there are some experiences at some points in time in some countries where participation did something useful. I would have hoped so. Unfortunately, you can drive a whole fleet of best practices between that and almost literally measuring the whole world against this yardstick. There is no real difference between somebody confusing a few countries' experience with NPM with budgetary best practice and your use of participation for the same end.

Let me stop here before getting into the SDGs makes this way too long. But I rather suspect that China will continue to be rather pleased with its development progress and can probably live comfortably in the knowledge that they might not fare well against the SDGs.

September 30, 2015 | Registered CommenterPhilipp Krause

From Paolo de Renzio, Senior Research Fellow, International Budget Partnership
I'm enjoying this, Philipp!

Ok, so let's leave aside the need for "handled-with-care" indicators, and the much bigger issue of China and the SDGs, at least for now, and let's focus on participation and accountability.

First of all, participation is also a right, we should not forget that. The right of people to be part of decisions that affect their lives is sanctioned in Art. 21 of the Universal Declaration of Human Rights, and in a number of other human rights treaties and covenants. However, in recent debates this has often been overlooked. Measuring and promoting participation therefore has intrinsic value for us, above and beyond participation’s instrumental function.

I totally agree with you that other aspects of social organization like a free press and competitive elections are fundamental to accountability. We do not claim to have the whole range of possibilities covered by our limited survey instrument. But we do try and cover most aspects of “budget accountability”, at least, and in our experience the combination of transparency and official oversight is often not sufficient to guarantee adequate accountability for the use of public resources, especially when oversight institutions suffer from some of the common weaknesses that the Open Budget Survey points to. It is not by mistake that the two countries identified as having the best functioning budget accountability systems in our 2013 book “Open Budgets”, Brazil and South Korea, both score very well on our budget participation questions. I agree with you that the evidence base is still very thin, and that’s an issue that that we constantly underscore, arguing for the need for additional research. Some of it is starting to appear, as shown in our “Impacts of Fiscal Openness” paper, and in other recent work that GIFT is sponsoring. But much more is needed to provide a solid foundation for both our arguments.

If we consider participation mostly as a right, then your argument that the fact that participation is not very widespread means that it probably doesn’t matter is clearly questionable. According to the same argument, one could say that the abolition of slavery or women’s right to vote were not important because they were not very commonly found at some point in time…

One thing I very much agree with you on: the danger of blindly promoting best practices, and the contradictory role that global indicators like the OBI can play in that regard. This is something we often discuss internally, and to which we have not yet found a full and satisfactory answer. When I present our findings, I often make the point that governments should experiment with different ways of doing things rather than follow our recommendations to the letter, and that – this language will sounds familiar – function matters much more than form. Yet, there’s a lot more we could and should do. Thanks for pushing us to continue to improve.


PS The “Open Budgets” book can be found here: http://www.brookings.edu/research/books/2013/openbudgets
PPS The GIFT papers on participation can be found here: http://www.fiscaltransparency.net/resources-all/

October 2, 2015 | Registered CommenterRyan Flynn

Hi Paolo,

So I think we're definitely getting to the heart of the matter here.

First off, you're right I fully agree that if you see participation as a right, then the fact that it's not widespread doesn't matter. And like with transparency, that would make it quite distinct from any utilitarian argument of "if you increase your participation score, you're more likely to get the following good effects".

The catch though is that I don't think there is a right to "participation" as you define it in the index, and it's certainly not directly prompted by Art. 21 of the Universal Declaration of Human Rights. What Art. 21 says is that "Everyone has the right to take part in the government of his country, directly or through freely chosen representatives", that people have equal access to public services and that there should be elections by universal suffrage. In other words, it expresses the right for people to have a stake in their government, and very clearly points to representation as a way to do so - you cannot seriously tell me that on this basis every country in the world is held against a standard that says "does your supreme auditor let the general public take part in the formulating its audit programme?"

The participation index develops a very specific and very particular and most importantly, to me, unhelpful model of how people can take a stake in public finance matters. It seems, again, to me, very clear that the much stronger route to popular accountability if you have reasonably transparent government is via a free press and effective representation that fills the legislature with a meaningful opposition challenger to the government.

As a citizen, I'd take a strong opposition or media challenging government statements on fiscal policy over some CSO participation mechanism any day. More importantly, as a citizen, I like my government representatives on their toes, but I also like them being able to do their jobs. I would be very dubious about my government establishing participation mechanisms for civil society, which might just be lobby groups for people whose priorities I don't share, because that devalues my vote. But of course that's just me, the bigger point here is that I don't think you can get from right to representation to your (As in IBP's) very specific measure of participation.

And this is where it gets tricky - because this accountability ecosystem is what is talked about and that's what IBP promotes. I do see a tiny bit of self interest involved when there have a group of CSOs defining participation in a way that very specifically involves giving CSOs a very direct role in government affairs. Nothing against self-interest (I wouldn't object to newspapers arguing for a free press, either), but it puts this in a certain light.

Which takes me back to the question of prevalence. I think one reason why the participation score is so low is not because the right to participation is in such a dire state, but because there are plenty of representative governments that use other mechanisms to realize people's rights just fine. To me, that needs to be part of the conversation when you work with governments and stakeholders. There will be cases, of course, where the thing to do is to organize participatory mechanisms in budget formulation. But there will be others where it's not, and that doesn't make those governments any less accountable.


October 2, 2015 | Registered CommenterPhilipp Krause

From Paolo de Renzio, Senior Research Fellow, International Budget Partnership

Hello (once again) Philipp!

Clearly when you get to the heart of the matter things get more interesting – and more complicated…

First of all, on participation as a right. While Art. 21 of the Universal Declaration is clearly – and with good reason – quite general, it does put the emphasis that it is not only through elections that citizens can and should participate in government affairs, but also through other “indirect” mechanisms. Over the years, this has been interpreted quite broadly, and incorporated in a raft of other human rights declarations, conventions and covenants. So our efforts are not an attempt to “stretch” human rights language in our favour, but they simply represent one possible application of the human rights legal framework to the realm of budgeting. In recent years, we have worked with the Global Initiative for Fiscal Transparency (GIFT) to develop a set of High-Level Principles for Fiscal Transparency, Participation and Accountability. The Government of Brazil took these Principles to the UN General Assembly for endorsement, which it did in December 2012. Principle 10 states:

“Citizens should have the right and they, and all non-state actors, should have effective opportunities to participate directly in public debate and discussion over the design and implementation of fiscal policies.”

Although I would love to think that this was all due to our self-interest and self-promotion, I doubt that that is the case. We are not that powerful or influent.

I understand your reservation on the particular way in which we try to operationalize the right to participation in budgeting through the Open Budget Survey, and I can assure you that we also don’t think we have got it right. We are again working with GIFT on revising the participation questions in the OBS questionnaire, to see how we can improve them. But your point on participation vs. free press and effective representation presents matters in an unhelpful either/or frame. Why should anyone choose between these different mechanisms for holding government accountable? Especially where a free press and effective representation are not present or very weak, as in many of the countries where the organizations supported by the IBP work, I’m sure many would take any other possible option over no option at all. At least that’s what we hear from our partners, for whom the Survey was originally designed.

For those governments in the Survey “that use other mechanisms to realize people's rights”, as you say, they can easily ignore our findings, and suffer no consequences. I would still want to check first, however, that their citizens also agree that their rights are being realized. If they don’t, our hope is that the Survey – as imperfect as it may be – can give them some ideas and arguments to get their governments to listen.


PS On how the UN approaches the issue of the right to participation, see for example:

And here’s the UNGA resolution on the GIFT Principles:

October 8, 2015 | Registered CommenterRyan Flynn

Hi Paolo,

I think we're agreeing on more and more things!

So let me get my agreements out of the way first. I totally buy that participation, broadly defined, is a good thing from a normative standpoint, although personally, I'd be much more willing to consider tradeoffs than with transparency. When you think about NIMBY-concerns around infrastructure provision, for instance, there is a balance to be struck between participation and effectiveness.

I also agree that if you want to measure and analyze participation, this index is a good place to start, although we seem to agree it could use a good amount of work (but as per our previous discussion, you'd expect that - all composite indices have issues).

Your comments above are essentially a defense of participation as a right worthy of consideration - no argument there. But that's not what the IBP report is saying. The IBP report isn't saying, "here's our transparency index, and here you have two other indices that you may want to consider, depending on the context".

The IBP report is, pretty much explicitly, looking at accountability and then working its way backward, saying "(budget) accountability is important, let's look at what is necessary to realize that. We're arguing you need a combination of transparency, participation and oversight".

But if you wanted to work your way backward from accountability, surely you'd have to look at all the important dimensions that make up (popular) accountability, and therefore need to include free press, competitive elections and a viable legislative opposition. Your argument that governments with free and open systems that don't have participation will do fine even if they score poorly on the combined indices doesn't hold water. Yes, there will be people working in less than free countries for whom participation is better than nothing, but these types of publications are influential and shape people's perception a lot - and to think that for accountability you need transparency, participation and oversight but not these other things is just wrong.

October 19, 2015 | Registered CommenterPhilipp Krause

PostPost a New Comment

Enter your information below to add a new comment.

My response is on my own website »
Author Email (optional):
Author URL (optional):
Some HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <em> <i> <strike> <strong>
« Benchmarking COP 21 – what could be the fiscal impact of fair carbon shares? | Main | Public Investment Management Assessments (PIMAs): Please Handle with Care »