The importance of values for substantiating evaluative conclusions

The comments shared in response to the earlier post, Culturally Competent Needs Assessment By An “Outsider” raise issues that are critical to the discipline of evaluation. Two things come to mind; a) reflections on how we define evaluation theory, and practice within the context of culture; b) the role of values and valuing in evaluation.

K. Fisher raised some critical questions about how we define evaluative culture in evaluation. It made me realize how we tend to think about culture, values, norms when we are about to evaluate a program, rather than ensure these things become an integral part of our ‘evaluation roots'; i.e., it needs to become part of the academic curriculum/syllabus. Without cultural competence, evaluators find themselves with quick ‘band aid’ solutions “to make the evaluation seem proficient to address cultural contexts issues”. There is a great need for scholarship in this area, particularly by evaluation institutions.

Addressing the issue of culture at the back end is not only unfair to the client, but also demeaning to the communities impacted by the program. Thanks David Earle, Jane and Patricia for putting a spin on this discussion. While diverse cultures exhibit ‘their own’ evaluative thinking, it is important to recognize these differences and learn from them.

The spin weaved here also touched on an important aspect of evaluation; i.e., the values and valuing. What is it that the community values? Is it intrinsically valuable and or desirable? What are the local norms, and the related dynamics that give rise to positive and or negative impacts to the evaluand? Whose reflections are we hearing? Which voices are not being heard? What is it that is not being said, and yet, it seems to matter?

If Betty LaDuke had not asked the insiders to help her ‘listen to their values, ‘speak their values language’ and ‘explore the deeper meanings behind the values’ that shape the day to day discussions, the artist would have walked away with unsubstantiated evaluative conclusions. Inclusion of values and how they were identified in the actual evaluation process has a bearing on the relevance and validity of the evaluation results.

Integration of ‘values’ and the relevant data collected during the process provided the basis on which the needs assessment (and its related components/ dimensions) had been judged as good or bad (merit), worthwhile or worthless (worth), and (significance).

Thanks for the suggestions My next post will discuss the dangers of evaluation that fails to ground itself in local realities.

3 comments to The importance of values for substantiating evaluative conclusions

  • Donna Mertens

    I agree with the sentiments raised by Tererai Trent and others re: this post. I situate myself in the transformative paradigm. This paradigm serves as an over-arching philosophical framework that begins with a belief in the need to be culturally responsive, respectful of cultural norms, aware of differences in power, and conscious of the need to bring voices of all stakeholders into the conversation in authentic ways. All stakeholders means that the evaluator is conscious of the voices of those who are marginalized in society in addition to those who are in the mainstream. While this positioning might be criticized as being about the politics of evaluation, I argue that without awareness of and responsiveness to political context, including cultural issues, we may well do a disservice to those who have the greatest potential for harm if we do it wrong.
    Cultural competence is not a state of being; it is an attitude of learning and respect that is dynamic and needs to be attended to in whatever context we are evaluating.
    Donna Mertens

  • k fisher

    I agree with you Tererai that it is unfair and demeaning to the client and communities if the issue of culture is only addressed at the ‘back end’ of evaluation theory and practice. It is for that reason that I think the evaluation community could take yet another step back to investigate and reflect on the cultural and historical specificity of what we do – including what currently constitutes culturally competent evaluative theory and practice.

    This is not because I think there are particular deficiencies in the practices that are being advocated in culturally competent evaluation (though I’m sure they’ll continue to evolve). It is because I think that by examining more closely what evaluators take for granted (eg like the importance of personal authenticity for truth) we can better recognise the ways in which cultural practices inescapably infuse our evaluative culture. I think this could help to sensitise us to levels and forms of difference (whether in the boardroom or village) that are not yet visible to us and that we have yet to articulate. I think there are two key advantages to this:
    1) It would help us to loosen the grip of our own cultural practice histories embedded in what we take for granted in evaluative theory and practice and so open ourselves up to new or existing ‘other’ ways of seeing, doing and thinking. At the very least, it may help us to respect existing ‘other’ ways as more equal or valid.
    2)It would help us avoid unwittingly masking or stomping on differences in the cultural practices of others. It expands the space in which we can recognise and make conscious choices about whether, and to what extent, we aim to influence and change the ‘different’ practices of others (culture and values being reconceptualised here as intrinsic to multiple, situated power-knowledge practices).

    I think this would be consistent with the ethos of acknowledgement of, if not automatic deference to, cultural and social difference. It also means we don’t have to assume that evaluative theory and practice can ever become a power-free, politics-free or a culturally neutral space.

    I think you can guess that I am persuaded of the Foucauldian orientation to knowledge and power and that I’ve also interpreted from the reply posts on culturally competent evaluation that, in the main, this orientation currently tends to draw more from the critical tradition of Habermas.

    Personally, I think by engaging with Foucault’s work and approach there is potential for evaluation to gain some important new perspectives that could inform the development of culturally competent theory and practice. I think his work is highly relevant because:
    1) it excels in methods for reflexivity at the level of social micro-practices;
    2) his work concentrates on examining the practices of Western culture and particularly those linked with the development of human science knowledges, techniques of government and techniques for governing the self (the domain of evaluation); and
    3) he brackets out questions about the value superiority or truth-status of difference.

    I’d be very interested in other people’s reaction to my comments and this suggestion.

  • David Earle

    Firstly, I am not a follower of Foucault because I have never managed to or been inclined to read his work. But I do understand that he opened up some important questions about social action, policies and power.

    Having thought a bit more about both of Tererai’s posts and our comments, I think one of the big silences in evaluation is discussion of the cultural and political nature of commissioned evaluations.

    There is a dangerous assumption that the evaluator is independent of the culture and values of both the commissioner and the evaluated – and able to draw on a set of culturally neutral methods for establishing value, merit and worth. And when the political nature of specific evaluations is raised in discussions, there is nervous laughter and a sense that was ‘that’ evaluation.

    What I think we need to discuss more is how we are continually implicated in the political and cultural agenda of our commissioners, at the same time as traversing the political and cultural agendas of those we evaluate. Both of whom have active agency in their worlds and quite often conflicting agendas. In the middle of this is our own personal integrity and life agenda – which may or may not line up with the rest.

    So cultural competency is not limited to working with the obvious “other” – but rather laying bare the culture of the commissioner, the evaluator and the evaluated, with a clear sense of the power relationships involved in all directions.

    In closing – I do agree that evaluators should examine and be informed by the ideas of Foucault and others. Currently I am just starting to reread Edward Said on imperialism and culture, as expressed through narrative fiction. Much of what he says can be applied equally to evaluation.

    I look forward to reading your next post, Tererai.