The risks of focusing on the easy-to-measure

Some more developments in UK development funding, an issue we looked at in a post a few weeks ago.

Lawrence Haddad, Director of the Institute for Development Studies has  an interesting article in the Guardian in response to David Cameron’s statements on international aid.

The best ways to deliver overseas aid are often not easy to measure

David Cameron’s plans risk diverting help away from where it’s really needed

Lawrence discusses David Cameron’s concerns about waste and corruption, with some telling comparisons to UK and USA examples.  He then discusses the risks in focusing on and rewarding only results that can be readily measured.

But my real worry about the new approach is that it will slant aid towards items on which it is easiest to demonstrate delivery. While the focus on “the things that aid can best deliver and that can make a real long term difference” sounds sensible, these very things may not be the most easily assessed.

The comments from readers are also interesting, including this one from “tiojo”:

I have a lot of respect for Mr Haddad’s institution. But he’s demonstrating why he’s an academic and not a practising aid worker. He ends his article by saying,

The most important spots that aid needs to hit may be the hardest to see.

Well Mr Haddad – you and your institution have been hard at work for many years now. Undertaking all the studies, research and evaluation. So tell us please – what are those hard spots that aid needs to hit and how do we hit them?

How do we balance the need to demonstrate progress (or lack of it) in readily understood measures or indicators, and the real challenges in measuring or describing long-term or diffuse outcomes such as community capacity which are also extremely important?

Thanks to Niels Keijzer from the Pelican Initiative – Platform for Evidence-Based Learning and Communication for Social Change for the link

3 comments to The risks of focusing on the easy-to-measure

  • Michael Scriven

    What the UK discussion about the important stuff that can’t be measured shows is that the Great Truth here is that the claim “the most important stuff is often not what we do or can measure” (or words roughly to this effect), is probably the most attractive unprovable claim in the evaluation game. It’s a bit like “the most attractive characteristic of some people is often one we can’t see.” These comments sound like insights but are in fact vacuous overgeneralizations from more useful claims like “Many evaluations of international aid programs fail to assess some of the most important indicators of value” which can and should be supported with examples, something you can’t do with the pseudo-insights (because if you could, that would falsify the claim).

    Cheers,

    Michael S

  • Dugan Fraser

    “These comments sound like insights but are in fact vacuous overgeneralizations”… Ouch! But of course he’s right. (That’s what happens when you’re Michael Scriven.)The role of the evaluator is to engage with the hard to measure stuff, to recognise and acknowledge the limitations…. not to cop out and say it can’t be measured.

  • Chad Green

    Thank you for sharing, Patricia. I watched the video from your other post and found this statement most salient: “Too often aid doesn’t make it to the people who need it.” This directly relates to David Cameron’s concerns, understandably, about waste and corruption.

    Personally I think the best use of aid is when both the donor and recipient directly benefit from the experience through a long-term partnership arrangement. This enables both parties to fully understand one another’s needs and perspectives on an equal footing.

    For example, as a former Peace Corps volunteer in Latvia (1994-96), I assisted with the growth of that country’s emerging NGO sector in Riga, as well secured funds to begin rebuilding their student government leadership capacity (formerly the Young Pioneer organization). I then returned to my home country with a different perspective and new skill set that could benefit my community. Coincidentally, it appears that such a proposal is in the works for the EU: http://www.guardian.co.uk/world/2010/mar/25/gordon-brown-european-peace-corps

    This is perhaps off topic, but can someone help me with this perplexing duality which I am currently trying to resolve: What is the synthesis of purity and purpose? I suspect the answer will differ from individual to individual, but for me the answer is our youth.

    Chad