Here in New Zealand we’re nearing the end of the central government financial year, with the usual utterly ridiculous frenzy of trying to jam just one more thing into this year’s budget before it gets sucked down a black hole never to be seen again. So, it seemed timely to come back to the notion of value for money – not of programs or policies, but of evaluations themselves.
There may well be some managers out there who have yet to experience the thrill of commissioning a total waste-of-money evaluation. Well, thankfully, your colleagues around the world have been working diligently on building a knowledge base just for you …
9 golden rules for commissioning a waste-of-money evaluation:
|1. Be vague about what kind of evaluation you need||There’s only one kind, isn’t there?|
|2. Be vague about the budget||It’s just the same as shopping around for widgets or fleet vehicles – price competition works exactly the same way.
And besides, evaluators are all mindreaders, so will easily figure out whether you want a Rolls Royce or a scooter.
|3. Use an onerous RFP process||Save on proposal reading time by narrowing the field down to large evaluation shops with minions who can copy and paste boilerplate and spend loads of time on a glossy 20+ page proposal.|
|4. Hire researchers to do an evaluator’s job||Oh, there’s a difference?|
|5. Opt for subject matter expertise over evaluation and cultural expertise||After all, evaluation’s just a content-specific measurement exercise, isn’t it?
Besides, there’s no such thing as “evaluation expertise”
And why would we need “cultural expertise” anyway? This program might be for a diverse community, but it’s not one of those “cultural” programs!
|6. Assume that “cultural expertise” is sufficient when the only brown faces on the team are the research assistants||Any whipper-snapper worth their salt can challenge a senior white evaluator and get them to completely reconceptualize a fundamentally flawed design.
Not that all that stuff matters anyway – it’s really just a matter of user-friendly faces to do the data collection and translation work.
|7. When budget is tight, opt for the contractor with the lower daily rate||After all, you can get more days/hours out of them, right?
Don’t be put off by pesky details, e.g. that an experienced evaluator working at twice the daily rate can get the job done 4 times as fast and to a level of quality that the less experienced person/team can never come close to no matter how much time they put in.
|8. Sign the contract, then hope for the best until the final report arrives||You don’t want to be breathing down the evaluator’s neck.
And besides, they know best what you need, for whom, when, why, and in what form.
|9. Bury and forget about disappointing evaluations||No point dwelling on a bad memory; just file it away and commission another one!|
Want to suggest a gem or two to add to the list? The comments box below awaits! Can we try for one from each region/continent here, just to ensure we are getting a global view?