Simple, complicated and complex perspectives on accountability and Three Cups of Tea controversy

I’m hopeful that the current controversy over Greg Mortenson’s book ‘Three Cups of Tea’ and the operations of the related NGO ‘Central Asia Institute’ (detailed in John Krakauer’s book Three Cups of Deceit and a 60 minutes story)  will lead to improvements in how development interventions and organizations are planned, implemented, evaluated and reported.  There are risks, however, that it could make it more difficult to raise funding in the areas of operation (girls’ schooling, Afghanistan, Pakistan).  And that it could provide further reasons for dismissing discussions about development evaluation that draw on complexity language.

In response to the criticisms raised by 60 minutes and by author,  Greg Mortenson, in a letter to supporters on the IKAT website, said:

Although we would like the world to be linear, orderly and peaceful, the reality is that our world is a dynamic, fluid place, often filled with chaos and confusion. In that space, I thrive and get the courage to help bring change and empower people. …

The “60 Minutes” program may appear to ask simple questions, but the answers are often complex, not easily encapsulated in 10-second sound bites.
Working in isolated areas, in communities that are not on any map, and often in areas of turmoil, religious extremism or natural disasters where education is still relatively rare and ancient codes of conduct and social hierarchies still dominate – all these things demand constant adjustment, accommodation and patience.

It’s interesting to analyze the case using the different lenses of simple, complicated, complex.

There appear to be simple aspects of this that should have been managed better .  Watch where the money goes, and make sure that most of it is going to the central activities (in this case, the description of the CAI on its financial accounts talks only about its international education programs, even though more money goes to the US ‘outreach’ book tours).  Check if the schools have been built, and are being used.

There appear to be complicated aspects that should have been managed better.  Schools need bulldings plus teachers plus course material.  Although these are listed as funded activities, it seems there has not been enough attention to ensuring the full ‘causal package’ is in place.

And, despite the language of complexity, there appear to be complex aspects that should have been managed better.  Since complex situations are unpredictable and emergent, it requires adaptive and responsive management.  Processes are needed to spot problems and work out how to fix them, and to spot successes and work out how to amplify them.  Instead, in Krakauer’s book, many staff and board members have recounted their experiences of finding problems and trying unsuccessfully to have them addressed.

Dealing with complexity effectively does not mean ad hoc or laissez-faire actions and reactions.  Nor does it mean ignoring the simple or complicated elements of the situation.

About a month ago, I wrote a post that wondered whether evaluation was really useful for accountability. The issues being raised about the Central Asia Institute have made me rethink this a little.  In that post I suggested that evaluation could usefully assess whether the management had displayed ‘smart accountability’ – being aware of what was going on, and adapting as necessary.  But, since most evaluators are not well trained or experienced in checking accountability for inputs, I suggested that auditors might do a better job of this.

However, the 2009 Audited Financial Statement for CAI, shows the limitations of audit processes:

Use of Estimates
Preparation of financial statements in conformity with accounting principles generally accepted in the United States of America requires the use of management’s estimates. Actual results of operations may differ from those estimates.

While evaluation needs to do more than see where the money has gone, how many activities have been undertaken, and how many outputs have been produced, it is important to have reasonable evidence of these – both for external accountability and to support informed, adaptive management that can truly respond to complexity.

4 comments to Simple, complicated and complex perspectives on accountability and Three Cups of Tea controversy

  • Patricia Rogers

    Saundra Schimmelpfennig on her blog ‘Good intentions are not enough’ has listed some lessons that should be drawn from this case.

  • I would note that there are audits… and then there are audits. Financial audits in general are never going to go beyond the kind of statement you quote. However, I would hope that a performance audit done properly would address multiple levels of complexity, and come closer to bridging the accountability gap you described in your previous post.

  • This particular case epitomizes why a certain segment of of blogosphere writes about the effectiveness of aid programs and charities. Most DIY aid organizations don’t like partnerships or collaborations because they are afraid of scrutiny. They want to create their own standards and rules to follow. Everyone wants to be a hero. The founders of these DIY organizations fear that someone else may get credit for their ideas and accomplishments. I call this the “Nobel Syndrome”. Being transparent might jeopardize their egotistical dreams of standing on a stage in Oslo and accepting the Nobel Peace Prize for their outstanding contributions toward humanity.

    These are some of my observations regarding the subject.

    Slactivism in Africa | Independent Global Citizen

  • Re: “there are audits…and then there are audits.”
    Several years ago, there was an effort in the Canadian audit community to deal with the issues in this post. They produced “12 attributes of effectiveness.” This work more-or-less disappeared as another flavour of the month, but the framework was far more useful–in my view–than a straight value-for-money perspective. The framework is still out there on the web in various places including in a draft paper “THE EVALUATION OF ORGANIZATION PERFORMANCE:

    However, I don’t think the framework really accounted for the non-linearity of complex systems. I did a workshop for accountants several years ago. It included a modified open space session in which one option was something like “What would accountability look like for complex environments and issues?” To my surprise, there were many people keen to explore that topic (but this was an isolated PD event with no follow up). So there’s hope.