Pushing sand uphill with a pointy stick? ‘No value-free’ in higher ed evaluation

Tweet

There’s a unique and extremely challenging barrier to singing the ‘no value-free’ parts of the genuine evaluation song in a higher education (a.k.a. tertiary education) setting.

And that’s what Michael Scriven calls the value-free doctrine.

Last week I delivered the opening keynote at the Self Assessment for Quality conference for tertiary (=higher) education organizations

Read the whole post –> Pushing sand uphill with a pointy stick? ‘No value-free’ in higher ed evaluation

Punished for productivity – poor use of an average in performance evaluation

Tweet

Jessica Miglio/NBC

Developing good performance indicators is not easy. The history of their use is littered with examples of how they can produce a distorted picture of performance and provide dysfunctional incentives. Burt Perrin’s report to the OECD (Organization for Economic Co-operation and Development) Implementing the vision – addressing challenges to results-focused management

Read the whole post –> Punished for productivity – poor use of an average in performance evaluation

Oxford admissions essay: “simple, yet devilish” … An evaluation aptitude test?

Many thanks to Michael Quinn Patton for sending us through this gem (from the New York Times) about a rather interesting essay exam for selecting graduate students into All Souls College in Oxford, England.

Read the whole post –> Oxford admissions essay: “simple, yet devilish” … An evaluation aptitude test?

What constitutes “evidence”? Implications for cutting-edge, tailored treatments, and small sub-populations

In the medical profession in particular, there are some very rigid beliefs about what constitutes good enough “evidence of effectiveness” to justify offering, recommending, allowing patients to try, or even just not vehemently opposing a particular type of treatment for a patient. There are some glimmers of hope in other sectors (e.g. in the Best Evidence Synthesis work here in New Zealand). But there are still three areas where there are very serious challenges in building a credible evidence base given the kinds of constraints and realities surrounding them. They are: (1) cutting-edge treatments; (2) treatments that are by their very nature tailored/individualized rather than standardized across patients or populations; and (3) learning what works for small sub-populations

Read the whole post –> What constitutes “evidence”? Implications for cutting-edge, tailored treatments, and small sub-populations

Bad faith survey

Over-sampling of particular population strata, and subsequent reweighting of the responses to match the population, might be appropriate sometimes, but not when it involves gathering and then discarding data about a politically contentious and high cost program.

Read the whole post –> Bad faith survey