Stuart Henderson’s mention of The Two Second Advantage (see the LinkedIn discussion referred to in the post from earlier this week) reminds me of the work of business strategist Arie De Geus, who discusses how learning organizations use scenario planning to create “memories of the future”.

It seems to me that this idea has potential not only in looking at emergent trends in evaluation (such as those Stuart is asking about on LinkedIn), but also as a lens we can apply to programs and organizations to help them consider how to ‘future-proof’ themselves.

De Geus points out that, although we can’t predict exactly what changes will happen in the future, it is possible to brainstorm a few different possible scenarios about what might happen.

Some scenarios will be more likely than others – some will seem almost certain; some will seem incredibly unlikely; some may incorporate natural disasters or substantial shifts in the world or in the community (either gradual or dramatic).

The scenarios may be developed – minimally or extensively, depending how likely the scenario seems – into contingency plans (i.e., What would we do if …).

The idea behind scenario planning is this:

• We know from psychology that the human brain is more likely to spot the early signs of change if the person has rehearsed the possibility beforehand. [I am sure some clever person has a citation for this!]

• Likewise, organizations are more likely to perceive change early – and have ideas for how to react effectively to take advantage of it – if they have gone through the process of dreaming up the various possible future scenarios.

Now, how might this help programs and organizations build strength and sustainability?

Think of the kinds of major and minor changes that could have major impacts on what the organization does or how it does it. Examples might include:

• A key funding source completely disappearing or dramatically reducing
• An immigration policy altering the demographics (and therefore the needs) of the population they serve
• A new technology enabling them to access remote areas, or making some aspect of their services obsolete
• A natural disaster such as an earthquake, volcanic eruption, or flood
• Two or three key people leaving the organization

Evaluations often have to answer questions about the sustainability, transferability, or applicability of programs, services, and activities over time, over space, over borders. One way of doing this is to help stakeholders think of possible scenarios and how well equipped the program or organization or community is to handle them.

Scenario planning is a tool that, as De Geus says, helps organizations create “memories of the future” that can help them detect the early signs of even unlikely change and to be better equipped to respond to it.

I am sure the folks involved in disaster-preparedness and emergency evaluation have a lot to say on this topic, as will those working in countries where political volatility is high and those with an interest in complexity and unpredictability – please chime in and share your thoughts!

So much to see, so many choices, and today I struck gold in session after session. Here are a few highlights from one of the best!

The indigenous TIG is one of the best-kept secrets at the AEA conference, and one of their offerings today was no exception. It was standing room only in one of the liveliest and most creative sessions of the conference, entitled The Evolution and Revolution of Culturally Responsive Evalation. On the creative end of things, we saw some brilliant of examples of metaphors and symbolism being used to effectively engage communities and to communicate concepts and findings. We also heard some great insights into culturally responsive evaluation – what it means in different settings, how it’s been developing over recent years, and the possibilities for the future.

Pam Frazier-Anderson, Stafford Hood, and Rodney Hopson talked about the challenges to translating evaluation concepts such as logic models into a form that African American communities could engage with. They presented the African American Culturally Responsive Evaluation System (ACESAS) and showed how they transformed a traditional boxes-and-arrows logic model into a sankofa bird, an African symbol representing the concept of reaching back into the past to fetch the strength with us into the future. This model  will be coming out in a forthcoming book on culturally responsive logic models.

Joan LaFrance gave an engaging presentation rich with imagery and metaphor, showing, with an amazing range of example,s how evaluations could be expressed as program stories within cultural metaphors. The key take-home messages here were that (1) evaluation fits within Native American cultures; it’s not just a Western or alien concept and (2) these approaches help Native American communities take ownership of evaluation for what it can do for them, not just as a requirement for the funder. There was a LOT to this presentation, and Joan mentioned that it will be uploaded onto the AEA e-library website (we’ll add a link when it is), and a book is available at http://iahec.org

Kataraina Pipi treated the audience to a beautiful waiata (song) to open, followed by a fascinating exploration of her and our take (in Aotearoa New Zealand) on culturally responsive evaluation. She contrasted responses from M?ori and non-M?ori evaluators to such questions as, When is it acceptable for evaluators outside the culture to say they are doing culturally responsive evaluation? and How do you understand your cultural position in your evaluation work within Maori communities? Kataraina commented on what a very exciting time this time and space is to be working on the development of culturally responsive evaluation.

An extremely interesting discussion followed Kataraina’s session when someone asked whether there were any problems with objectivity when M?ori evaluators worked on evaluations within M?ori communities. Nan Wehipeihana explained that, in Aotearoa (New Zealand), credibility comes not from independence or distance, but from connectedness – for more details on this take, see also: Credibility and independence in evaluation – an alternative view

The final presentation was a creative series of two skits performed by Jennifer Greene, Jeehae Ahn, and Ayesha Boyce that highlighted how assumptions about people of different cultures and ethnicities can exist on either the client side or the evaluator side, and must be explored via ‘responsible responsiveness’ to develop contextualized understandings of diversity and advance equity in both opportunity and accomplishment.

Discussants Karen Kirkhart and Nan Wehipeihana drew out some interesting themes in the session. Karen noted that all theory is culture, athough it’s not always named. She also made the link back to Michael Scriven’s Tuesday workshop on reconceptualizing evaluation, arguing that the kinds of thinking and examples we are seeing emerge in this space are, in her view, something of a Copernican revolution. Nan pulled out a theme across the sessions about the importance of knowing who you are and where you stand, so that you also know who and where you are not. She also highlighted the value and challenges of working in deliberately cross-cultural teams where people ask the difficult questions and really challenge each other, in a collegial way.

We found this pic on Funny Craig's List Ads - click pic to see source

Having spent last week with people chiming in from all around the world to discuss cultural context and evaluation, we are wondering if our readers would be up for a collaborative all-continents (and as many countries as possible) effort to internationalize this US-originating classic that has done the rounds all over the Internet:

How To Identify Where A Driver Is From

[We found this version at commonplacebook.com]

1. One hand on wheel, one hand on horn: CHICAGO

2. One hand on wheel, one finger out window: NEW YORK

3. One hand on wheel, one finger out window, cutting across all lanes of traffic: NEW JERSEY

4. One hand on wheel, one hand on newspaper, foot solidly on accelerator: BOSTON

5. One hand on wheel, one hand on nonfat double decaf cappuccino, cradling cell phone, brick on accelerator, gun in lap: LOS ANGELES

6. Both hands on wheel, eyes shut, both feet on brake, quivering in terror: Ohio, but driving in CALIFORNIA

7. Both hands in air, gesturing, both feet on accelerator, head turned to talk to someone in back seat: ITALY

8. One hand on 12 oz. Double shot latte, one knee on wheel, cradling cell phone, foot on brake, mind on radio game, banging head on steering wheel while stuck in traffic: SEATTLE

9. One hand on wheel, one hand on hunting rifle, alternating between both feet being on the accelerator and both feet on brake, throwing McDonald’s bag out the window: TEXAS

10. Four-wheel drive pick-up truck, shotgun mounted in rear window, beer cans on floor, squirrel tails attached to antenna: ALABAMA

11. Two hands gripping wheel, blue hair barely visible above windshield, driving 35 on the Interstate in the left lane with the left blinker on: FLORIDA

12. Republican sticker on bumper, turning left on a no-left-turn intersection, kids in back seat screaming and flipping off other drivers: INDIANAPOLIS, INDIANA

Help us internationalize this guide for a global audience with your observations from your native country and anywhere else you have lived or visited!  We are aiming for something from as many continents and far-flung places as possible.

Tererai meeting with former US president Bill Clinton

We had some great conversations here last week when Zimbabwe-born California-based evaluator Dr. Tererai Trent joined us as guest blogger to spark a week of lively discussion on the topics of evaluation and cultural context. We had such an enthusiastic and engaged response to Tererai’s contributions that we are planning to have her back again sometime soon to share some more of her unique insights.

In the meantime, those of you who enjoyed the discussion will be delighted to hear that Tererai has been inspired to start her own blog: Tinogona Evaluation. You can sign up for Tererai’s email or RSS feeds by going to the blog site. We’ll be following along!

For more information about Tererai or to get in touch with her directly, check out her website, Tinogona Consulting. Tinogona means “It is achievable” in the Shona language of Zimbabwe. And for those who haven’t seen the TV snippet of her amazing story on the Oprah website, we highly recommend it!

We also recommend checking out the Interdisciplinary Ph.D. in Evaluation, where Tererai earned her doctorate alongside a number of other incredibly talented evaluators making their mark in evaluation today. You may well meet some more of them guest blogging here on Genuine Evaluation in the months to come!

“Given the norms that govern most patriarchal societies in Africa, should the Western epistemology, ethics and concepts be the main default lens for evaluation” “Despite their blindness to social cultural context, are these evaluations valid even though they are said to be based on scientific evidence”

A, B, and C—the ways of HIV/AIDS preventing transmission

Uganda, Adaptation of the Fleet of Hope, 1995. (click image to go to source article)

Figure – Social Marketing to Prevent HIV/AIDS: Uganda, Adaptation of the Fleet of Hope, 1995. (Jhuccp.org)

I conducted a metaevaluation on a number of evaluations from some of the most prominent bilateral agencies fighting HIV/AIDS in Sub Sahara Africa. Despite insufficient evidence to support evaluation claims, the majority of the evaluations assessed include recommendations which indicate either a need for continued funding, and/or increase of monitoring staff.

Here are some of the selected examples of my findings;

Troubling Question

The critical question of concern is where is the value judgment underlying recommendations found in some evaluation reports which fail to seek for side effects

 Conclusion

Obvious the basis upon which HIV/AIDS prevention interventions maybe too narrow to shift the potential underlying social ecology (negative gender dynamics and socio-cultural norms) that gives rise to women’s HIV/AIDS vulnerability. The western lens may not be the main default lens in evaluation; social cultural and gender norms are an important part of the landscape for Sub-Saharan Africa. These dynamics and norms represent a Pandora box entrenched with hidden contents (i.e., side effects) which affect gender and HIV/AIDS prevention.

Watch for my next post, where I will be discussing the hidden contents of the Pandora’s box which are being ignored by evaluation.