Shelfari: Book reviews on your book blog

Evaluation and complexity

I’ve recently been running workshops on purposeful program theory in Australia, New Zealand and the United States. It’s been a great treat to explore with so many different people how we might develop, represent and use program theory for policies and programs that have significant complicated or complex aspects

It’s an issue I’ve

Read the whole post –> Evaluation and complexity

Multimedia report on 'Show me the change' conference

Wonderful interactive report on the recent ‘Show me the change’ conference in Melbourne on complexity, behavior change and evaluation.

An open space conference such as this has magic moments of connection and creation. Documenting them through videos, notes, reflections and cartoons helps to capture the moment for participants and others.

How to spot a 'lip service' approach to culturally responsive evaluation

So you’ve put out an RFP for an evaluation of a policy, program or initiative intended to serve and effect positive change in a “minority” community. All the proposals look terribly impressive, and they all include “cultural experts” on the evaluation team. How can you distinguish the proposals that show a clear understanding of what it takes to do effective and culturally responsive evaluations from those that merely pay ‘lip service’ to cultural competence?

Read the whole post –> How to spot a ‘lip service’ approach to culturally responsive evaluation (a checklist for evaluation commissioners)

Evaluation revisited - conference this week on complexity and evaluation

If you can’t be in the Netherlands this week for the conference on complexity and evaluation, you can follow it through the conference blog

Read the whole post –> Evaluation revisited – conference this week on complexity and evaluation

£6 million over 5 years - and STILL no genuine evaluation of Blueprint?

When a large and expensive evaluation fails to produce useful results, it’s worth seeing if at least it can be useful as a cautionary tale.

Blueprint is a UK Government-funded drugs education programme consisting of five components: drug education in schools (for 11 and 12-year-olds – this was the main emphasis), drug education for parents,

Read the whole post –> £6 million over 5 years – and STILL no genuine evaluation of Blueprint?

Show me the change - a conference on complexity and evaluating behaviour change

Interesting conference in terms of content and processes in Melbourne May 4-6 at the Abbotsford Convent:

Behaviour change and evaluation are both complex and fascinating fields. With an increasing awareness of the need to implement community-wide change towards a more sustainable society, there are numerous endeavours working to change people’s behaviours. But how do we know

Read the whole post –> Show me the change – a conference on complexity and evaluating behaviour change

Learning from failure

Being able to learn from failure is an important part of Genuine Evaluation. A recent paper by Cannona and Edmondson explores technical and organizational barriers to doing this. Are there good examples of addressing these in monitoring systems or evaluation

Read the whole post –> Learning from failure

The Friday Funny: "Right Answers"

As evaluators we all have to struggle, on the one hand, with having at least some sense of what evidence should constitute “good performance” (for a program, policy, or whatever we are evaluating) … but on the other hand, with being sufficiently open-minded about examples of good – perhaps outstanding – performance that we hadn’t imagined ahead of time and that clearly don’t fit with the program designer’s or funder’s plan.

On that topic, this urban legend, plucked from mathematical finance academic Mark Joshi’s academic jokes page, “courtesy of Kevin Lim (15/01/04)”, tickled our funny bones and seemed an apt reflection for this week

Read the whole post –> The Friday Funny: “Right Answers”

A fleshed out 'program logic' for why and where 'insiders' are included in evaluation

**Revised and updated, March 4th** Here’s an elaborated version of the table presented in our earlier post that discussed the implicit reasons for creating evaluation teams with cultural (and other) ‘insiders’ in different proportions and in different roles. The table is presented in four columns:
* Implicit “Problem” or “Challenge” Addressed
* Insider Inclusion Rationale
* Likely Practice Implication (How ‘Insiders’ Are Involved)
* Likely Evaluation ‘Product’

Read the whole post –> A fleshed out ‘program logic’ for why and where ‘insiders’ are included in evaluation

Supporting evaluators in cross cultural contexts

The concept of (invited) guest or gatecrasher was introduced by Judy Oakden as a way of framing her role, place and space in Maori focused evaluation as a non-Maori evaluator. In this post Nan Wehipeihana addresses several thought-provoking questions, including:
* How then do I come to invite non-Maori to be part of a mainly Maori evaluation team?
And in a mainly Maori evaluation team:
* What are the roles non-Maori evaluators play?
* What are the responsibilities when inviting non-Maori evaluators?
* What are the challenges of involving non-Maori evaluators?
* What are the benefits of involving non-Maori

Read the whole post –> Supporting evaluators in cross cultural contexts: Invited guests or gatecrashers?