Who’s afraid of the Big Bad … Thought?

At the recent conference of the Australasian Evaluation Society, Tom Schwandt gave an entertaining and thought-provoking keynote in which he talked about a societal phenomenon called ‘phronemophobia’. [Tom assured us this really was an existing word in the English language and not something he made up!]

Phronemophobia is the fear of thinking, which Tom says manifests itself in both thinking paralysis and minimalist thinking. An evaluative example he gave was monitoring performance against taken-for-granted benchmarks – as opposed, presumably, to posing the right questions and then thinking about what blend of evidence and evaluative interpretation will be required to generate answers.

I have also found this to be a recurring theme in the assumptions that pervade not just the client universe we work in, but also the thinking of the many professionals out there who engage in evaluative work but who often don’t identify as ‘evaluators’. It also pervades the ‘thinking’ of the media and (therefore) the general public.

Sadly, we quite often see this in graduate-level evaluation training.

People often seem to think evaluation is all about measures and metrics, and that once you have them collected, summarized, graphed, and tabled, somehow the end product is “an evaluation.” Somehow it is assumed that the data, the metrics, ‘speak for themselves’.

The evaluative interpretation step is frequently missing from reports that are presented as ‘evaluations’. And the ‘how to’ surrounding this all-important slice of evaluation-specific methodology is conspicuously absent from much evaluation training.

Tom puts this slightly differently, and made several very important points in his keynote:

  • Data aren’t ‘evidence’ until they are assembled in defense of a particular claim or argument.
  • Evaluation textbooks should – but don’t – cover an analysis of important cognitive biases and how they affect evaluative judgment. [Yes! See the excellent ‘cognitive bias’ videosong from an earlier post here on the blog.]
  • We need to get clearer about what constitutes good evaluative judgment. “Good judges should get it right and think right.”
  • Evaluation training needs more critical thinking. [Totally agree! And not just in that, but also in the logic, methodology, and skills of evaluative reasoning.]

Thoughts? … if that’s not too frightening a thing to do …

3 comments to Who’s afraid of the Big Bad … Thought?