I often see the terms monitoring and evaluation used in the same breath, and have heard many comment that M&E is usually much more M than E.
It seems to me that the lack of a clear distinction between the two means that evaluation is getting shortchanged.
So, what is the difference?
Monitoring and evaluation ask and answer very different kinds of questions – and therefore need different methodologies to generate the answers to those questions.
|Coverage||Monitoring Question Examples||Evaluation Question Examples|
|Outputs (Products, Services, Deliverables, Reach)||How many people or communities were reached or served?Were the targeted numbers reached?||How adequate was program reach?Did we reach enough people? Did we reach the right people?|
|Process (Design & Implementation)||How was the program implemented?Was implementation in accordance with design and specifications?||How well was the program implemented?
Fairly, ethically, legally, culturally appropriately, professionally, efficiently?For outreach, did we use the best avenues and methods we could have?How well did we access hard-to-reach and vulnerable populations?Did we reach those with the greatest need?
Who missed out, and was that fair, ethical, just?
|Outcomes (things that happen to people or communities)||What has changed since (and as a result of) program implementation?How much have outcomes changed relative to targets?||How substantial and valuable were the outcomes?How well did they meet the most important needs and help realize the most important aspirations?Should they be considered truly impressive, mediocre, or unacceptably weak?Were they not just statistically significant, but educationally, socially, economically, and practically significant?
Did they make a real difference in people’s lives?
Were the outcomes worth achieving given the effort and investment put into obtaining them?
The need for the program – taken as given, or not?
Another key difference is that the need for the program is generally assumed in monitoring, which basically asks whether a program is on time, on target, and on budget.
In evaluation, it is also part of our job to ask about the need for the program. e.g., “Was the program – and is still – needed? How well does it address the most important root causes? Is it still the right solution?”
After all an on-target program that is no longer needed isn’t a good or worthwhile one. And evaluation, by definition, would say so. With monitoring that larger “question the very existence of the program” element is not generally part of the brief.
Want to know more?
I ran a short webinar recently on how to frame high-level questions to guide an evaluation (and how to distinguish these from monitoring questions). I’m planning to create a tutorial on this topic, so if you’d like to hear when this is coming up, please join my free newsletter. And, coaching is available if you’d like hands-on help to apply it to your own work.
I’m also planning some tutorials on evaluative rubrics, which can be developed independently (by the evaluation team) or collaboratively (with community members, program staff, and/or other stakeholders), and used to interpret quality and value in a systematic and transparent way, for qualitative, quantitative, and mixed method evidence.
For those looking for a low-cost resource to explain these concepts, I have an easy-to-read minibook out that explains the key distinctions, as well as some of the methodologies needed to answer evaluative questions (e.g. not just “what are the outcomes” but “how good are the outcomes”).
It’s called Actionable Evaluation Basics: Getting succinct answers to the most important questions [minibook]. Check the reviews on Amazon to see what other people thought – and then add your own review!
The Spanish version is also available on Amazon as an ebook (with many thanks to Pablo Rodriguez-Bilella for the translation work!). Please help guide other potential buyers by writing a thoughtful review of it in Spanish – much appreciated!
UPDATE: French version coming soon (Feb 2014): Les essentiels de l’évaluation tournée vers l’action: Obtenir des réponses succinctes aux questions les plus importantes. To be first to hear when it’s published (as a paperback and ebook), join my free newsletter.