The importance of visible, high level commitment to evaluation

One of the favorite stories I tell about evaluation is about going to meet a senior manager to discuss evaluation and finding him standing on a table in the middle of an open-plan office, with the staff gathered around him, as he stretched up to his full 6 foot 5 inch height , thumped one hand with the other for effect and exclaimed “Evaluation is IMPORTANT!”

It’s become my benchmark for visible, sincere and enthusiastic support from senior management for evaluation.

This week that manager, Dr Bruce Kefford, now Deputy Secretary, Agriculture Research and Development in the Victorian Department of Primary Industries, spoke to the meeting of the Victorian Branch of the Australasian Evaluation Society about developments in evaluation since that time. He is still enthusiastic about evaluation, and talked about the strategies they have put in place to prevent or reduce the common problems of evaluation such as goal displacement and data corruption.

He also referred to the 7 lessons we drew from the first 5 years of their work to build organizational evaluation capacity and capability (capacity is like having a water tank; capability is having pipe connected to it so you can make use of the capacity – thanks to Bob Williams for the metaphor). These were published in the article Bron McDonald, Patricia Rogers, and Bruce Kefford Teaching People to Fish? Building the Evaluation Capability of Public Sector Organizations Evaluation January 2003 9: 9-29:

  1. start small and grow evaluation;
  2. address both supply and demand;
  3. work top-down and bottom-up simultaneously;
  4. use a theory of change behaviour;
  5. develop a common evaluation framework, including a generic programme theory;
  6. build knowledge of what works within the agency’s context; and
  7. systematically and visibly evaluate each stage.

Looking back at the list, and thinking about many of the other organizations I’ve worked with over the years, I think many of these still stand up as good advice. I’m not so sure a generic programme theory is always helpful (they found they got more traction using a common template, but there are obvious risks if the template doesn’t suit all the types of programs).

What is missing is the very important strategy of trying to reduce the fear of evaluation by making it clear that identifying problems did not always leads to closing programs (maybe they could be improved), or where it did, that this did not always lead to losing staff (maybe they could move to another program).

Another issue that is not addressed is the importance of sustained effort over time. Some organizations were shocked to see that the process had taken 5 years (and of course is still continuing), and wanted instead a way they could just get it all done in a couple of months.

What do you think of the others? Are they visible in other successful efforts to grow evaluation capacity and capability?

4 comments to The importance of visible, high level commitment to evaluation