Shelfari: Book reviews on your book blog

The importance of visible, high level commitment to evaluation

One of the favorite stories I tell about evaluation is about going to meet a senior manager to discuss evaluation and finding him standing on a table in the middle of an open-plan office, with the staff gathered around him, as he stretched up to his full 6 foot 5 inch height , thumped one

Read the whole post –> The importance of visible, high level commitment to evaluation

How much evidence is needed for policy?

In the last few days before the Australian federal election, a curious $5million advertising campaign has been launched which claims to be advocating evidence-based policy but does nothing of the

Read the whole post –> How much evidence is needed for policy?

Managing genuine evaluation paradoxes: Genuine reporting

In reponse to the earlier post on genuine evaluation snippets from around the globe, Irene Guijt raised a very important question about the tensions between several hallmarks of genuine evaluation:

Some important contrasts presented but also one that doesn’t entirely align – tell the whole story but cut to the chase? Include activities,

Read the whole post –> Managing genuine evaluation paradoxes: Genuine reporting

More utterly uncritical media reporting of evaluation

As an evaluator, or even as an interested member of the public, what very basic fundamental information would you expect to see in a newspaper article that kicked off like this?

Chiefs get good marks even if departments don’t

Public sector leaders are mostly embracing challenges thrown to them by the Government, though at least one is “lost at sea”, a report says.

The Trans Tasman Media report, which ranks government agencies and bosses, has found that chief executives are generally performing better than their departments.

Read the whole post –> More utterly uncritical media reporting of evaluation

'Ripping off the cloak of secrecy

New UK Government commits to “ripping off the cloak of secrecy”. How long will this thirst for transparency last?

Read the whole post –> “Ripping off the cloak of secrecy” – British PM pledge

What constitutes "evidence"? Implications for cutting-edge, tailored treatments, and small sub-populations

In the medical profession in particular, there are some very rigid beliefs about what constitutes good enough “evidence of effectiveness” to justify offering, recommending, allowing patients to try, or even just not vehemently opposing a particular type of treatment for a patient.

There are some glimmers of hope in other sectors (e.g. in the Best Evidence Synthesis work here in New Zealand). But there are still three areas where there are very serious challenges in building a credible evidence base given the kinds of constraints and realities surrounding them. They are: (1) cutting-edge treatments; (2) treatments that are by their very nature tailored/individualized rather than standardized across patients or populations; and (3) learning what works for small

Read the whole post –> What constitutes “evidence”? Implications for cutting-edge, tailored treatments, and small sub-populations

Friday Funny - the front fell off

This week’s Friday Funny illustrates the importance of being able to admit and discuss failure (or mixed success): It’s an Australian-New Zealand collaboration YouTube video called ‘The Front Fell Off’.

Read the whole post –> Friday Funny – the front fell off

£6 million over 5 years - and STILL no genuine evaluation of Blueprint?

When a large and expensive evaluation fails to produce useful results, it’s worth seeing if at least it can be useful as a cautionary tale.

Blueprint is a UK Government-funded drugs education programme consisting of five components: drug education in schools (for 11 and 12-year-olds – this was the main emphasis), drug education for parents,

Read the whole post –> £6 million over 5 years – and STILL no genuine evaluation of Blueprint?

Analyzing failure in criminal justice reform

The Center for Court Innovation has embarked on a project that seems very relevant to our recent discussions about learning from failure (or from less than total success):

The Center for Court Innovation, with the support of the U.S. Department of Justice’s Bureau of Justice Assistance, has embarked on a multi-faceted inquiry designed to analyze failure

Read the whole post –> Analyzing failure in criminal justice reform

Advocating for RCTs - with a non-RCT example?

Curious post by Tim Harford in the Financial Times recently “Political ideas need proper testing” that slides from advocating for better empirical investigation of public policy by systematic experimentation to discussing this only in terms of RCTs – and then uses as the exemplar a brilliant example of using other types of evidence to inform

Read the whole post –> Advocating for RCTs – with a non-RCT example?