Old mistake in today’s article on European responses to austerity measures – here, as reported by Karen Kissane in The Age in Melbourne:

Meanwhile, in Greece, a country spiralling into poverty with more than half of its young people unemployed,

This seems to be a common misinterpretation. A Reuters report led with the headline:

Over half of Greek youth unemployed

A BBC report stated:

More than half of young people in Greece are unemployed …

Well. no, actually.  Unemployment rates are the percentage of those available for and looking for work who are unemployed.  Since a large proportion of young people are engaged in full time study, the proportion of the population who are unemployed is much less.

According to the European Commission’s Eurostat page “Statistics explained” on unemployment statistiscs, Greece’s youth unemployment rate has gone up from 25.7% in 2009 to 44.4 in 2011 and 49.3 in the last quarter of 2011 – but the ratio – that is, youth unemployed as a percentage of the youth population has only increased from 8% in 2009 to 13% in 2011.  Still a tragedy for those affected, but very different to the picture painted.

Discussions about public policy need to be based in accurate representation of the situation. This is a basic mistake that should not be made.

A couple of years ago we quoted a paper on psychological research called “Keeping it simple” (Peterson & Park, 2010) that observed:

… the evidence of history is clear that the research studies with the greatest impact in psychology are breathtakingly simple in terms of the questions posed, the methods and designs used, the statistics brought to bear on the data, and the take-home messages.

In the post, Simplicity and Genuine Utilization, Jane lamented our tendency in evaluation to overcomplicate things thanks to our training in the [social] sciences.

Nevertheless, we do realize that some of our colleagues are also in the business of communicating not to normal people (clients) but to editors and reviewers of academic journals who have the power to publish them or let them academically perish.

For the benefit of our evaluation colleagues in academia, here is a selection of snippets from Science Magazine columnist Adam Ruben, author of Surviving Your Stupid, Stupid Decision to Go to Grad School … Enjoy!

How to Write Like a Scientist

(click the title above to read Adam Ruben’s full post on the Science Magazine website – we’re posting just a few tantalizing snippets here)

2. Using the first person in your writing humanizes your work. If possible, therefore, you should avoid using the first person in your writing. Science succeeds in spite of human beings, not because of us, so you want to make it look like your results magically discovered themselves.

4. The more references you include, the more scholarly your reader will assume you are. Thus, if you write a sentence like, “Much work has been done in this field,” you should plan to spend the next 9 hours tracking down papers so that your article ultimately reads, “Much work has been done in this field^{1,3,6-27,29-50,58,61,62-65,78-315,952-Avogadro’s Number}.” If you ever write a review article, EndNote might explode.

9. Starting sentences with “obviously” or “as everyone knows” demonstrates your intellectual superiority. If possible, start sentences with, “As super-intelligent beings like myself know,” or “Screw your stupidity; here’s a fact-bomb for you.”

10. Your paper will be peer reviewed, so include flattering descriptions of all of your peers. Scientists call these “shout-outs” or “mad props.”

12. If you’re co-authoring a paper, most of your notoriety will derive from the order of authors and not from the content of your paper — so make sure to have vehement and petty debates about whose name goes first. Here are the general rules for authorship:

FIRST AUTHOR: Weary graduate student who spent hours doing the work.

SECOND AUTHOR: Resentful graduate student who thinks he or she spent hours doing the work.

THIRD AUTHOR: Undergraduate just happy to be named.

FOURTH AUTHOR: Collaborator no one has ever met whose name is only included for political reasons.

FIFTH AUTHOR: Postdoctoral fellow who once made a chance remark on the subject.

SIXTH AUTHOR: For some reason, Vladimir Putin.

LAST AUTHOR: Principal investigator whose grant funded the project but who hasn’t stood at a lab bench in decades, except for that one weird photo shoot for some kind of pamphlet, and even then it was obvious that he or she didn’t know where to find basic things.

Related posts:

• Simplicity and Genuine Utilization
• Managing genuine evaluation paradoxes: Genuine reporting

References/further reading:

Keeping it simple: Christopher Peterson and Nansook Park on the lasting impact of minimally sufficient research. The Psychologist, Vol. 23(5), 2010, pp. 398-401.

Unlearning some of our social scientist habits: Jane Davidson on how academic training in the social sciences can impede genuine evaluation. Journal of Multidisciplinary Evaluation, 4(8), 2007, pp. iii-vi.

In Genuine Evaluation we focus a lot on asking the right questions, bringing an evaluative frame, and basing answers on sound evidence.  But effective communication is also an important part of genuine evaluation, which is why this video caught our eye.  Hat tip to Stephanie Evergreen on twitter (@evalu8r) for sharing this.

Pic by JitterBuffer

Over the Antipodean summer Genuine Evaluation goes to the beach instead of blogging.  We’re back now, brushing off the sand, and planning more discussions about what it means to do genuine evaluation, plus sharing some insights from the African evaluation conference in Accra, Ghana.

To start the year, we wanted to highlight one of the more disturbing aspects of public policy discussions in recent years  -  the tendency to put forward opinions as if they were as compelling as solid evidence. We suspect that this will be the first in an ongoing series of examples.

Are working hours getting longer? Hopefully this example reflects someone being misquoted in the article in The Age in Melbourne, rather than how it appears – a researcher suggesting it’s too hard to get reasonable estimates of the extent of a problem and then pronouncing that the problem has diminished:

Some recent studies suggest this may now be a relic of history and that Australians work the longest hours in the developed world.

But Professor Mark Wooden, of the Institute of Applied Economic and Social Research, disagrees … strongly.

“The idea we work the most hours in the world is absolute crap,” he says.

“Lots of people work long hours and lots of people work short hours. We have a mix.”

He argues that workers in Japan and Korea work longer than Australians and that comparing working hours between countries was an inexact science.

People tend to overestimate how long they work as a sort of “badge of courage” and find it difficult to estimate the hours they work accurately, Professor Wooden says.

“I don’t think we can count,” he says. “It’s impossible to know. The study would need to be so invasive.”

His research shows the number of Australians working 50 hours a week or more peaked in the mid-1990s.

“Over the last 10 years, the proportion of Australians working long hours has been dropping.”

Photo: Joe Armao (The Age)

What’s worse than no evaluation? An evaluation that is wrong but you think is right.

Organizations that provide authoratitive evaluations have an obligation to meet high standards of accuracy and consistency. It is therefore hard to believe the series of events that led to Victoria’s Environmental Protection Agency listing Melbourne’s bayside beaches as “good”, and suitable for swimming when the level of bacteria was 40 times the acceptable limit.

According to The Age report,

[EPA] staff were not working on the ChristmasDay and Boxing Day public holidays, and as a result old forecast information was fed automatically into the website from Saturday until yesterday. These forecasts were made on Friday afternoon, and did not take into account the ferociousness of the Christmas Day storms.

On Sunday (Christmas Day) a massive storm hit Melbourne, sending debris, rubbish, cigarette butts and dog droppings into storm water drains. But for two days the EPA was operating on autopilot, issuing reports on the web site and tweets based on the projected water quality not the actual water quality.

And what has the EPA learned from this? Nothing, apparently. According to The Age, no EPA staff will be working on the New Year’s Day public holiday, and once again reports will be based on the weather forecast not on actual testing. A spokesman said beachgoers “should use their own judgment” in deciding to go swimming after storms like those that hit on Christmas Day.

Which does raise the question – if beachgoers should use their own judgment, because it is more likely to be accurate than the official reports, what’s the point of having the official reports?

Here’s what SHOULD happen. Either roster someone to work on these public holidays and ensure the reports are actually based on data ( it’s summer here and people are going to the beaches) OR issue a clear statement on the website, tweets and to news media that an accurate report cannot be provided due to the public holidays. No report is better than an inaccurate report.