Where and Why Western lenses miss the mark in Africa: The case of HIV/AIDS prevention evaluations

“Given the norms that govern most patriarchal societies in Africa, should the Western epistemology, ethics and concepts be the main default lens for evaluation” “Despite their blindness to social cultural context, are these evaluations valid even though they are said to be based on scientific evidence”

 

 

A, B, and C—the ways of HIV/AIDS preventing transmission

Uganda, Adaptation of the Fleet of Hope, 1995. (click image to go to source article)

Uganda, Adaptation of the Fleet of Hope, 1995. (click image to go to source article)

Figure – Social Marketing to Prevent HIV/AIDS: Uganda, Adaptation of the Fleet of Hope, 1995. (Jhuccp.org)

—“do not have sex” and “stick to your partner, or else, use a life jacket (condom) if you fall off the boat the chances of death are very high” Says Evaluation default lens.

I conducted a metaevaluation on a number of evaluations from some of the most prominent bilateral agencies fighting HIV/AIDS in Sub Sahara Africa. Despite insufficient evidence to support evaluation claims, the majority of the evaluations assessed include recommendations which indicate either a need for continued funding, and/or increase of monitoring staff.

Here are some of the selected examples of my findings;

Criticism Evidence, Explanation, & Reasoning
Lack of Consideration of gender as an evaluative criterion Despite the indisputable data on gender disparities, gender as an evaluative criterion is not considered (e.g., assessment of gender specific interventions which may indicate gaps in prevention).
Lack of Conceptualization of Cultural Values, Sexual Behavior as it relates to HIV/AIDS (as a poor health outcome) There was lack of conceptualization of the meaning of “behavior,” in relation to HIV prevention, and yet, to a larger extent culture determines ones’ behavior.
Too much focus on Quantitative Measures

 

Measures were mainly in terms of “behavioral practices and changes”. The behavioral indicators were not assessed to determine their relevence and effectiveness. While the HIV incidence as an outcome measure was never assessed, the efficacy and the validity of the link between changes in specific behaviors and the potential for reductions in HIV incidence—the ultimate goal of prevention interventions—are not very clear.
Seeking Attribution drives the Use of Log frames at the expense of more congruent causal inference approach.

 

The Causal analysis mainly focused only on “behavioral changes” and failed to research different pathways of causality and their multiple relationships in HIV prevention. Entrenched gender norms and culture are common in social interactions, yet the causal mechanisms are hidden to an outside evaluator.  Inadvertently, these planned and intended outcomes of the program abandon any unintended outcomes (either positive or negative) and the side effects.
Lack of Evaluation Capacity and Independence

 

Too often evaluation methodologies and designs employed are largely supplied by donor agencies. Most donor agencies use the TORs to prescribe the “what and how” of evaluation methodology, which may affect the quality and robustness of the evaluation. This led to evaluation methodologies mostly prescribed around program goals, and only measure indicators spelled out in the program/project log frame, without assessment of other impacts, and or search for side effects including unintended consequences.
Poor Assessment of Institutional Processes to Assess “True Outcomes”

 

The evidence indicates poor assessment of institutional process-oriented criteria, whose results are important as feedback mechanism for institutions to strengthen processes (such as gender specific interventions; advocacy, policy measures, knowledge and leadership cultivation, and evidence based M&E).

Troubling Question

The critical question of concern is where is the value judgment underlying recommendations found in some evaluation reports which fail to seek for side effects

 Conclusion

Obvious the basis upon which HIV/AIDS prevention interventions maybe too narrow to shift the potential underlying social ecology (negative gender dynamics and socio-cultural norms) that gives rise to women’s HIV/AIDS vulnerability. The western lens may not be the main default lens in evaluation; social cultural and gender norms are an important part of the landscape for Sub-Saharan Africa. These dynamics and norms represent a Pandora box entrenched with hidden contents (i.e., side effects) which affect gender and HIV/AIDS prevention.

Watch for my next post, where I will be discussing the hidden contents of the Pandora’s box which are being ignored by evaluation.

6 comments to Where and Why Western lenses miss the mark in Africa: The case of HIV/AIDS prevention evaluations

  • Michael Patton

    Tererai,

    Your thoughtful and important example highlights one of the greatest weaknesses in much evaluation: recommendations unconnected to findings.
    Evaluators get lots of training in data gathering and analysis, but almost none in how to generate meaningful, appropriate, evidence-based recommendations.

    Thanks for sharing your insights.

    Michael

  • Jane Davidson

    Yes, I totally agree with Tererai and Michael on this one.

    Put another way, it’s the almighty logical leap from “What’s so” (descriptive data) to “Now what” (recommendations), but skipping the all-important “So what” (evaluative interpretation) step in between the two.

  • david earle

    I think Michael and Jane are both right. I also think we need to go deeper on this one. Following on Tom Schwandt’s fear of thinking, we also have the fear of engagement with the people using the service (I am aware of so many educational evaluations that fail to meaningfully capture the learner voice).
    But more insidious is a fear of making a difference. I really wonder if many evaluators (and I will include myself in this at times) are quite comfortable with the barely, hardly, never used evaluation report – and don’t want to enter a space where they may be seen as accountable for the changes arising from their work. Which would in turn expose their situation within interconnected political systems.
    It is not just that it is difficult to tell a funder that their favourite project is a head horse – but also fear that the funder might act on this knowledge and make drastic, possibly adverse, changes. It is far easier to keep everyone happy with a waffly report that fails to engage people in the real issues. Unfortunately the horse is still dead and the clients still not served.

  • Heather Young-Leslie

    I agree with David: In my experience with some NGOs, there is such a corporate anxiety with assuaging the donors’ concerns and not appearing ineffective, that real, reflective “so what” and “how make a real difference” (have an impact) questions appear risky. Because, the fear goes, the donor might withdraw the funds and then *nothing* would be done, and *nothing* is worse than a ‘dead horse’. (which may or may not be true)

  • Morton Juma Kimungui

    Hi Tererai,
    I have been reading your findings with keen interest, you have really convinced me.
    Am the executive director of Center for Integrity Promotion and Information (Kenya)
    From the analysis of the baseline data and the additional data gathered through focus group discussions concerning governments and donor funds as well as development in East Africa especially Kenya.
    Corruption is still the main impediment and extensively noticed awful translating to your findings. Political loyalties have led to unfair sharing of resources. In addition, You will also concur with me that, be it civil society organs or private sector even Prelates today minimize integrity unless closely monitored, they diverts funds for selfish gains. There is a general lack of transparency and accountability probably due to the blending of supervisory and implementing roles.
    Poor awareness by community members who should participate in viable project identifications has contributed to poor performance and in some cases a complete failure of the donor funds.
    It has been my opinion that donors revise their tradition system of funding intermediaries and liaise with community level structures aware of circumstances affecting their livelihood as part of their participatory knowledge to why billions of dollars funded has changed little since independence.
    Donors must sensitize communities explain why; everyone should take the fight against official corruption personally and see the act as the threat to his / her hopes for better life. Information is fundamental, furthermore at the core of human rights system will enable communities to exercise their rights, asses when their rights are at risk and determine who is responsible for any violation.
    kiminguimort@yahoo.com

  • ANDEBO PAX PASCAL

    Dear Tererai,

    I have just read your comments on evaluation of development projects. I find it with a lot of insights. As one involved in some development activities in education, I notice that your assessment cuts across. I will be grateful to read more of these comments in future; they provide a good guide to monitoring and evaluating the education programs I am involved in. Thanks too to the friends who have posted ideas here. They have helped me see M&E from a more practical perspective than theory which looks a straight line.