Tag Archives: RCTs

Some thoughts from Day 1 of the AEA conference

A couple of highlights from the opening sessions of the American Evaluation Association’s annual conference (being held this year in sunny San Antonio, Texas): Opening plenary Three views on evaluation quality, the theme of this year’s conference, from Eleanor Chelimsky, … Continue reading

Posted in Appropriate inference, Meta-evaluation, Synthesis of findings | Tagged , , , , | Leave a comment

What constitutes “evidence”? Implications for cutting-edge, tailored treatments, and small sub-populations

In the medical profession in particular, there are some very rigid beliefs about what constitutes good enough “evidence of effectiveness” to justify offering, recommending, allowing patients to try, or even just not vehemently opposing a particular type of treatment for a patient.

There are some glimmers of hope in other sectors (e.g. in the Best Evidence Synthesis work here in New Zealand). But there are still three areas where there are very serious challenges in building a credible evidence base given the kinds of constraints and realities surrounding them. They are: (1) cutting-edge treatments; (2) treatments that are by their very nature tailored/individualized rather than standardized across patients or populations; and (3) learning what works for small sub-populations Continue reading

Posted in Causal inference, Community programs, Development, Education, Government programs, Health, Strategic policy evaluation | Tagged , , , , , , , , , , | 12 Comments

The Friday Funny: A review of RCTs on parachute use

We recently stumbled across this all-time classic that Genuine Evaluation readers may well appreciate!

Smith, G. C. S. & Pell, J. P. (2003, December). Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomised controlled trials. BMJ, 327, 1459-1461. Continue reading

Posted in Causal inference, Friday Funnies, Health | Tagged , , , , | 1 Comment

Advocating for RCTs – with a non-RCT example?

Curious post by Tim Harford in the Financial Times recently “Political ideas need proper testing” that slides from advocating for better empirical investigation of public policy by systematic experimentation to discussing this only in terms of RCTs – and then uses as the exemplar a brilliant example of using other types of evidence to inform policy. Continue reading

Posted in Causal inference, Causal inference strategies, Government programs | Tagged , , | Leave a comment

Investing In Innovation – a need to apply what we know about evidence-based policy

The new funding rules for the US Department of Education’s $650 million Investing in Innovation appear based on an out-of-date model of evidence-based policy and hierarchy of evidence. Recent developments in our understanding of evidence-based policy would suggest changes are needed to the selection criteria and to how successful proposals will be evaluated. Continue reading

Posted in Causal inference, Causal inference strategies, Education | Tagged , , , , , , , , , | 4 Comments