Jane posted the first draft of my proposed workshop for the summer institute at Claremont Graduate University, where I suggest it’s time to reconceptualize evaluation from the ground up. Here’s a little bit more to explain what I’m after with that theme…
Let’s define a Copernican revolution—somewhat along the lines of ‘paradigm shift’—as a radical shift in the framework of our thinking about a substantial subject matter area, i.e., a rejection and/or redefinition of the most fundamental assumptions involved in the theories and possibly the language and data formats of the area. In the physical sciences, geocentrism was an early example of a framework that got rocked, and absolute space another. In these terms, I want to say that the discipline of evaluation as it now stands has gone through two of these already. The first was the shift away from the basic concept of program evaluation that was part of those texts in the social sciences that mentioned it at all in the 1950s/60s era, namely the idea that evaluation consisted of: (i) formulating the goals of the program in behavioral terms, (ii) finding or creating a test of those behaviors, and (iii) applying the test to the targeted population so that one could determine how good the program was by finding out the extent to which it had met these goals. You still see professional evaluators giving that as the definition of evaluation on Evaltalk and (more often) elsewhere. It involves half a dozen gross errors, including the exclusion of goal critique, ethical process critique, side-effect determination, cost analysis, comparisons, etc. (See Hard-Won Lessons in Program Evaluation… an issue of New Directions in Program Evaluation, as it was then called.) It was replaced by a much more far-reaching paradigm, hinted at in the phrase ‘determining comparative cost-effectiveness within an ethical/pragmatic framework’.
The second Copernican revolution in evaluation, on my account, was the shift from the ‘geocentric’ fallacy of thinking that evaluation is program evaluation—exhibited in the titles of textbooks and theories/models which announce their coverage of evaluation but with a content discussing only program evaluation—to the ‘heliocentric’ approach that takes the key focal element to be evaluation as the sun around which rotates a solar system of planets including program evaluation, product evaluation, policy analysis, personnel evaluation, etc. This shift leads to many practical as well as theoretical/conceptual results of great importance, like the first shift, but details of those is not my present topic.
Now, I’m suggesting that it’s time to be more self conscious about those past shifts, in our thinking and practice AND to consider whether it’s time for a third shift. What might that be? Suggestions from you—and criticism of the above account—would now be appropriate, and comments by you on those proposals, possibly including some from me if I’m not knocked unconscious by the brilliant insights themselves!
P.S. (A comment about e.g., transformative evaluation.) The shift away from assuming traditional Western European values to a framework more accepting of other cultural norms including those which fully accept the rights of women, ethnic, religious, GLBTG, and ‘disabled’ persons is part of the ethical framework referred to in the second revolution above, although it would certainly be an important revolution in its own right if there weren’t some other radical considerations that must also be incorporated.