I’m hopeful that the current controversy over Greg Mortenson’s book ‘Three Cups of Tea’ and the operations of the related NGO ‘Central Asia Institute’ (detailed in John Krakauer’s book Three Cups of Deceit and a 60 minutes story) will lead to improvements in how development interventions and organizations are planned, implemented, evaluated and reported. There are risks, however, that it could make it more difficult to raise funding in the areas of operation (girls’ schooling, Afghanistan, Pakistan). And that it could provide further reasons for dismissing discussions about development evaluation that draw on complexity language.
In response to the criticisms raised by 60 minutes and by author, Greg Mortenson, in a letter to supporters on the IKAT website, said:
Although we would like the world to be linear, orderly and peaceful, the reality is that our world is a dynamic, fluid place, often filled with chaos and confusion. In that space, I thrive and get the courage to help bring change and empower people. …
The “60 Minutes” program may appear to ask simple questions, but the answers are often complex, not easily encapsulated in 10-second sound bites.
Working in isolated areas, in communities that are not on any map, and often in areas of turmoil, religious extremism or natural disasters where education is still relatively rare and ancient codes of conduct and social hierarchies still dominate – all these things demand constant adjustment, accommodation and patience.
It’s interesting to analyze the case using the different lenses of simple, complicated, complex.
There appear to be simple aspects of this that should have been managed better . Watch where the money goes, and make sure that most of it is going to the central activities (in this case, the description of the CAI on its financial accounts talks only about its international education programs, even though more money goes to the US ‘outreach’ book tours). Check if the schools have been built, and are being used.
There appear to be complicated aspects that should have been managed better. Schools need bulldings plus teachers plus course material. Although these are listed as funded activities, it seems there has not been enough attention to ensuring the full ‘causal package’ is in place.
And, despite the language of complexity, there appear to be complex aspects that should have been managed better. Since complex situations are unpredictable and emergent, it requires adaptive and responsive management. Processes are needed to spot problems and work out how to fix them, and to spot successes and work out how to amplify them. Instead, in Krakauer’s book, many staff and board members have recounted their experiences of finding problems and trying unsuccessfully to have them addressed.
Dealing with complexity effectively does not mean ad hoc or laissez-faire actions and reactions. Nor does it mean ignoring the simple or complicated elements of the situation.
About a month ago, I wrote a post that wondered whether evaluation was really useful for accountability. The issues being raised about the Central Asia Institute have made me rethink this a little. In that post I suggested that evaluation could usefully assess whether the management had displayed ‘smart accountability’ – being aware of what was going on, and adapting as necessary. But, since most evaluators are not well trained or experienced in checking accountability for inputs, I suggested that auditors might do a better job of this.
However, the 2009 Audited Financial Statement for CAI, shows the limitations of audit processes:
Use of Estimates
Preparation of financial statements in conformity with accounting principles generally accepted in the United States of America requires the use of management’s estimates. Actual results of operations may differ from those estimates.
While evaluation needs to do more than see where the money has gone, how many activities have been undertaken, and how many outputs have been produced, it is important to have reasonable evidence of these – both for external accountability and to support informed, adaptive management that can truly respond to complexity.