Hot tips for commissioning and managing actionable evaluation

Michael Quinn Patton gave an interesting keynote address this morning on how evaluation needs to change in order to align with the changed and ever-changing world we live in. But how well has the evaluation commissioning process – and particularly the procurement guidelines in government) kept up with this idea?

What can we learn from both stunningly high value evaluative work (“dream projects”) and bitter disappointments (a.k.a. “Nightmares on Eval Street”)? In an AES presentation this afternoon, I drew on these experiences – and some thoughts about our changing, complex world – as a foundation for a “hot tips” guide for those who commission evaluation – and the evaluators who work with them.

The slides are embedded below, for those interested (click through to the post on the site if you can’t see them):

There was a lively discussion with audience members from both the client side and the evaluation contractor side.  Perhaps readers of Genuine Evaluation will have some thoughts to add?

As a follow-up on a presentation, I promised I would post this audio interview/conversation I did recently with colleague Rire Scotney, who has considerable experience working on the client side. She and I discuss an innovative approach to commissioning evaluation, in a way that meets the central government procurement requirements, but works quite differently from the usual default. Enjoy! [Again, those of you on the email or RSS feed may need to click through to the post to access this audio.]

2 comments to Hot tips for commissioning and managing actionable evaluation

  • Allison Titcomb

    Brilliant! Thanks so much for sharing your wisdom with those of us unable to be there with you in person.

  • Hi Jane. Looks like Adelaide is brilliant. I have just listened to the interview again. Thank you for the opportunity to participate in that with you. It was fun. Well done for managing the technology too!

    There is a point in there that is worth drawing out. That is: when is evaluation thinking useful? I would say that it is just as useful at the beginning of an initiative as it is at the end. A wise client would spend money up front to understand the very early indicators of “what good looks like” (and for that matter “what not good looks like”). This is not as easy as it sounds as some of these indicators can be well hidden in their early stanges. A well constructed “theory of change” map or intervention logic diagram would go a long way. Even this, though, needs to be broken in to the smallest pieces of information or indicators, that become evidence as it unfolds, not just as it was discovered in hindsight.

    So I am saying evaluation can be used to manage something that’s still going. Not just to find out about it afterwards. I am sure the evaluation community knows this only too well. It’s us clients that need to get with it more.