Sincerity in evaluation – highlights and lowlights

image from

image from

Principles of Genuine Evaluation

When we set out to explore the notion of ‘Genuine Evaluation’, we identified 5 important aspects of it:

  1. VALUE-BASED -transparent and defensible values (criteria of merit and worth and standards of performance)
  2. EMPIRICAL – credible evidence about what has happened and what has caused this,
  3. USABLE – reported in such a way that it can be understood and used by those who can and should use it (which doesn’t necessarily mean it’s used or used well, of course)
  4. SINCERE – a commitment by those commissioning evaluation to respond to information about both success and failure (those doing evaluation can influence this but not control it)
  5. HUMBLE – acknowledges its limitations
From now until the end of the year, we’re looking at each of these principles and collecting some of the highlights and lowlights  from 2010 (and previously).

The centrality of sincerity

Sincerity of evaluation is something that is often not talked about in evaluation reports, scholarly papers, or formal presentations, only discussed in the corridors and bars afterwards.  And yet it poses perhaps the greatest threat to the success of individual evaluations and to the whole enterprise of evaluation.

Are only new governments sincere about evaluation?

1. Victoria
Over the weekend, my home state of Victoria had an election which resulted in a change of government.  The new government has come in promising transparency:

‘There will be no hidden agenda, no spin, no secrecy. Accountability and transparency will the principles that underpin our government,’ Mr Baillieu said on Tuesday.

Funnily, the previous government when it came to power in 1999 made all the same claims.  Let’s see if the new government can walk the talk once it is ensconced in the government seats.
2. USA Federal Government
Meanwhile, the Obama administration is also promising greater transparency on its Transparency and Open Government site:
Government should be transparent. Transparency promotes accountability and provides information for citizens about what their Government is doing.  Information maintained by the Federal Government is a national asset. My Administration will take appropriate action, consistent with law and policy, to disclose information rapidly in forms that the public can readily find and use. Executive departments and agencies should harness new technologies to put information about their operations and decisions online and readily available to the public. Executive departments and agencies should also solicit public feedback to identify information of greatest use to the public.

3. Indian National Government

In India, a new independent evaluation office has been established that will focus on assessing the impact of government’s flagship programs (thaks to Denis Jobin for this news):

The Union Cabinet today approved the establishment of an Independent Evaluation Office (IEO) to undertake impartial and objective assessment of the various public programmes and improve the effectiveness of public interventions. This is in pursuance of the Presidential address to the Joint Session of both Houses of Parliament in June, 2009 to establish an Independent Evaluation Office at an arms’ distance from the Government to assess the outcomes and impact of the major flagship programmes of the Government of India.

The IEO will be an independent office attached to the Planning Commission under a Governing Board chaired by the Deputy Chairman, Planning Commission. It will be funded by the Planning Commission and will have, as its head, a full-time Director General (DG) in the rank and status of Member, Planning Commission. It will have full functional autonomy to discharge its functions. The IEO will also advise the Planning Commission and the implementing agencies in developing appropriate management systems consistent with the evaluation objectives.

The IEO will strengthen the existing evaluation process by drawing the best resources available from leading research organizations. The findings of the independent evaluation will be reported to the Government of India and also be placed in the public domain.

4. UPDATE New York City’s Mayor’s Office of Operations.

( thanks to GreeneBarrett on twitter) -A searchable display of indicators about agency performance, and overall graphics.

(It does not however provide any commentary on them.  For example, the Brooklyn Public Library is way under its targets in terms of opening hours.  Is this due to lack of staff (or other factors the program should be managing) or heavy snow (or other factors the program might not be able to influence)? In the new year we will return to this and other performance reporting sites to compare them to good practices for performance reporting)

Are there other examples of goverments promising transparency – or even continuing to deliver on this several years down the track?

Comments are closed.