How to kill a discipline: Worship the theorists, diss the implementors & the evaluators!

Michael ScrivenA blast from the past that has not lost its relevance, and that comes back to me again and again.

I did love Michael Scriven’s analysis in his 2013 keynote to the Australasian Evaluation Society in Brisbane, where he remarked that the academic theorists are always considered the high priests of any discipline; the pure experimentalists closest/2nd tier; the “applied” folks, including the evaluators, have always been the almost-outcasts, their work sullied by getting their hands dirty in the real world.

But, as Scriven says, this attitude (and hierarchy) will be the death of the social science disciplines themselves.

Why? Because the most important questions any discipline ever asks and answers are the applied – especially the evaluative – questions. Like:

  • What’s the most effective way to bring an economy back from a recession?
  • What’s the most powerful way to transform social, educational, cultural, environmental, & economic systems so that they are sustainable and support thriving communities/economies/individuals?

The more these “applied” avenues of thought and inquiry are dismissed, the more a discipline relegates itself into the annals of irrelevance. Eventually resulting in “death by irrelevance” (of the discipline).

The only salvation:

  • The applied researchers & evaluators should be at the “core” of any discipline.
  • Theorists as satellites in service of the applied work.

But let’s take Scriven’s idea another step.


The theorist, the implementor, and the evaluator

Let’s start with the first two …

The theorist comes up with a notion about how the world works, or could be made better.

The “implementor” is seen as merely the person who puts the theorist’s brilliant ideas into practice.

What people hugely underestimate, though, is that just as much intellectual horsepower has been plunged into thinking through how exactly to contextualize an abstract notion and bring it to life in the real world with tools, systems, ways to persuade people and help them understand, support to help them do it, how to explain it to hostile stakeholders, and (of course – some of the evaluator’s piece) how to know in real time what’s working, what’s not, how we know, and what needs fixing first and fast!

Practical application (implementation) is in fact theory creation in its own right. From theory-based practice to practice-based theory. Adding to the theory what the theoretician never thought of because they don’t have that grounding in how it actually works on the ground. Heck, a lot of great theory is built from the ground up – how do you think we invented theory in the first place?

The implementor is the chef who takes the food theorist’s recipe, tries it in the kitchen, and works out how to bring out flavors to die for! The result is not just a vastly improved recipe; it now has the crucial added dimension of know-how (general and local), how to put “the twist” on the idea in just the right place to give it the edge.

The evaluative piece is just as important. It’s not just how we track things, people. It’s part of how we bring the theory to life in the real world. It’s how we show people “This is what success looks (and tastes) like when it happens; this is how you’ll know.” It’s about shared vision, something that guides the chefs, the sous-chefs, and the entire kitchen; it helps them visualize and therefore create it in real time, to the absolute delight of the clients.

Why do we privilege certain kinds of knowledge and insight creation over others that add just as much to the mix?

What happens when we do?

We get recipe books that look impressive but produce substandard cuisine. They knew too little about the nuances of the ovens or the local ingredients, or too little about the clientele and what they really yearned for.

Theorist: We love you and your brilliance, but you can’t make magic on your own.

Your implementor and your evaluator are brimming with just as much genius, just a different variety from yours.

And that’s what makes us lethal together.


2 comments to How to kill a discipline: Worship the theorists, diss the implementors & the evaluators!

  • Thank you!! On several occasions, I’ve heard people say “we need an academic researcher” for an applied research project. Without even considering an applied researcher as a possibility. And then when they get the results they complain that the research was too academic-focused and not applied-focused enough.

    In the field of physics, scientists have made breakthrough discoveries through theoretical physicists and applied physicists working together. Similarly, for social science to produce results that are highly useful for solving important issues, we need collaboration between theoretical and applied social researchers.

  • Pete McMillen

    I can think of at least one social science (largely reinvented as ‘science’ in a strictly quantitative sense) where collaboration between the theorist and empiricist is fundamentally flawed. Specifically, economics. Or more precisely neoclassical economics. Whose largely hokum theories remain centre stage orthodoxy thanks to vested interests’ hegemony and influence in writing off so much of the immense volume of empirical evidence (the ‘evaluators’) that challenge their fairy tale theories of perfect competition and market equilibrium, perfect information, strictly rational and self-interested behaviour, forever downward sloping demand curves, inneficiency and distortions when governments intervene to correct their fallous claim that market failure is rare, etc. It might however be prudent to continue dissing the implementors of neoclassical inspired interventions.