|
||||||
 Many thanks to Michael Quinn Patton for sending us through this gem (from the New York Times) about a rather interesting essay exam for selecting graduate students into All Souls College in Oxford, Read the whole post –> Oxford admissions essay: “simple, yet devilish” … An evaluation aptitude test? In the medical profession in particular, there are some very rigid beliefs about what constitutes good enough “evidence of effectiveness” to justify offering, recommending, allowing patients to try, or even just not vehemently opposing a particular type of treatment for a patient. There are some glimmers of hope in other sectors (e.g. in the Best Evidence Synthesis work here in New Zealand). But there are still three areas where there are very serious challenges in building a credible evidence base given the kinds of constraints and realities surrounding them. They are: (1) cutting-edge treatments; (2) treatments that are by their very nature tailored/individualized rather than standardized across patients or populations; and (3) learning what works for small Read the whole post –> What constitutes “evidence”? Implications for cutting-edge, tailored treatments, and small sub-populations  Over-sampling of particular population strata, and subsequent reweighting of the responses to match the population, might be appropriate sometimes, but not when it involves gathering and then discarding data about a politically contentious and high cost program. Read the whole post –> Bad faith survey When a large and expensive evaluation fails to produce useful results, it’s worth seeing if at least it can be useful as a cautionary tale. Blueprint is a UK Government-funded drugs education programme consisting of five components: drug education in schools (for 11 and 12-year-olds – this was the main emphasis), drug education for parents, Read the whole post –> £6 million over 5 years – and STILL no genuine evaluation of Blueprint? Languaging (finding ways for difficult or complex ideas to make sense in different contexts) is a very important issue for getting people to buy into (and take action based on) evaluation findings, particularly when some aspect of a program is not doing well. Positive languaging can be highly effective for getting stakeholders to buy into not-so-positive findings. However, we do need to be wary of defaulting to positive, ‘comfort zone’ language all the Read the whole post –> Languaging in evaluation – raising fewer hackles vs. clarity of message The new funding rules for the US Department of Education’s $650 million Investing in Innovation appear based on an out-of-date model of evidence-based policy and hierarchy of evidence. Recent developments in our understanding of evidence-based policy would suggest changes are needed to the selection criteria and to how successful proposals will be Read the whole post –> Investing In Innovation – a need to apply what we know about evidence-based policy Is certainty of measurement the most important criterion for impact evaluation? Colin Burrows has set forward a tongue-in-cheek proposal for measuring the impact of research undertaken in universities – the Impact-o-meter. This satirical piece raises serious questions about the cost, precision and implications of measuring impact. As an aside, Australia had developed a framework for assessing Read the whole post –> The Impact-o-meter Most lay people can grasp the difference between grading/rating and ranking, so what’s wrong with the media? Following on from Patricia Rogers’ recent posts about the misreporting of evaluation findings, this post looks at an example from the New Zealand media (reporting on the new National Standards for literacy and numeracy) of leading the public astray with a complete lack of understanding of this very fundamental evaluation concept. Jane also ponders the reasons why the mainstream media in particular gets this kind of thing wrong so often … Read the whole post –> The media and evaluation reporting – clueless or unscrupulous? Another Head Start evaluation, another controversy about whether the results show it works or not. In her comment on our post on the NY School Milk Study Susan Wolf drew our attention to some important differences between the recent evaluation report on Head Start, and how it was represented in an email from the Brookings Read the whole post –> Does the recent evaluation show that Head Start doesn’t work? |
||||||
|
Copyright © 2010 Genuine Evaluation - All Rights Reserved |
||||||
Recent Comments