Very excited to see that Kylie Hutchinson and James Coyle have posted their long-anticipated interview with Michael Scriven!
Welcome to a tour of the cosmology of evaluation. This time we’re going by train, and Michael Scriven has some news for us!
“Guys, you all got on a train and you didn’t check the destination. You just checked the question of what it was first going to get to, which was program eval, and then there were little hamlets along the way, like personnel eval and so on.
But actually, this train is going to follow what the definition of evaluation is in the dictionary because that’s what it chose to call itself.
And I have news for you. It’s got some pretty remarkable places where it’s going to stop and you are going to have to show your pass. And we expect you to be doing a little work on the voyage, on the trip, towards making yourself able to get yourself through the pass control system.”
What are those destinations? Here are two you should know about:
- Intradisciplinary Evaluation. No area of study can even call itself a discipline without sound evaluation of everything it does. No, this doesn’t just mean evaluation of policies and programs relevant to the discipline. This is about how the discipline decides what makes a significant vs. a trivial advance in its discipline, a good vs. a flawed theory, a high vs. low quality piece of research, a worthy faculty member, a good dissertation, and so forth. As evaluators, we have a duty to contribute to this, to help other disciplines make sure they get it right.
- Ethical analysis. No, not just “research ethics”, but the more difficult task of determining whether the policy/program/project/product itself and its outcomes are ethical. At the trickiest end of the spectrum, it might involve working out whether a “good” women’s health clinic should include pregnancy termination services.
What else are you going to need to get through the “pass control system” Michael mentions?
Evaluation-specific methodology. These are the methodologies that are both distinctive to and essential for evaluation – but that most evaluators do not yet have in their toolkits.
And don’t forget to sign up for the podcast on iTunes as well!
While you’re there, be a good evaluator and rate the podcast series.