I often see the terms monitoring and evaluation used in the same breath, and have heard many comment that M&E is usually much more M than E.
It seems to me that the lack of a clear distinction between the two means that evaluation is getting shortchanged.
So, what is the difference?
Monitoring and evaluation ask and answer very different kinds of questions – and therefore need different methodologies to generate the answers to those questions.
|Coverage||Monitoring Question Examples||Evaluation Question Examples|
|Outputs (Products, Services, Deliverables, Reach)||How many people or communities were reached or served?Were the targeted numbers reached?||How adequate was program reach?Did we reach enough people? Did we reach the right people?|
|Process (Design & Implementation)||How was the program implemented?Was implementation in accordance with design and specifications?||How well was the program implemented?
Fairly, ethically, legally, culturally appropriately, professionally, efficiently?For outreach, did we use the best avenues and methods we could have?How well did we access hard-to-reach and vulnerable populations?
Did we reach those with the greatest need?
Who missed out, and was that fair, ethical, just?
|Outcomes (things that happen to people or communities)||What has changed since (and as a result of) program implementation?How much have outcomes changed relative to targets?||How substantial and valuable were the outcomes?How well did they meet the most important needs and help realize the most important aspirations?Should they be considered truly impressive, mediocre, or unacceptably weak?
Were they not just statistically significant, but educationally, socially, economically, and practically significant?
Did they make a real difference in people’s lives?
Were the outcomes worth achieving given the effort and investment put into obtaining them?
The need for the program – taken as given, or not?
Another key difference is that the need for the program is generally assumed in monitoring, which basically asks whether a program is on time, on target, and on budget.
In evaluation, it is also part of our job to ask about the need for the program. e.g., “Was the program – and is still – needed? How well does it address the most important root causes? Is it still the right solution?”
After all an on-target program that is no longer needed isn’t a good or worthwhile one. And evaluation, by definition, would say so. With monitoring that larger “question the very existence of the program” element is not generally part of the brief.
Want to know more?
I ran a short webinar in December on how to frame high-level questions to guide an evaluation (and how to distinguish these from monitoring questions). I’m planning to run this again (an improved version – in short and long formats), so if you’d like to hear when this is coming up, please join my free newsletter.
I’m also planning some webinars on evaluative rubrics, which can be developed independently (by the evaluation team) or collaboratively (with community members, program staff, and/or other stakeholders), and used to interpret quality and value in a systematic and transparent way, for qualitative, quantitative, and mixed method evidence.
For those looking for a resource to explain these concepts, I have an easy-to-read and low-cost minibook out that explains the key distinctions, as well as some of the methodologies needed to answer evaluative questions (e.g. not just “what are the outcomes” but “how good are the outcomes”). “Actionable Evaluation Basics: Getting succinct answers to the most important questions [minibook]” Check the reviews on Amazon to see what other people thought – and then add your own review!
At the moment this is available as an ebook on Amazon (which you can read on a PC, Mac, Smartphone or Kindle)
I am also nearly ready to publish a Spanish ebook version (with many thanks to Pablo Rodriguez-Bilella for the translation work!), so stand by for that if Spanish is your preferred language.
UPDATE: The Spanish version is now available on Amazon as an ebook! Readable on PC, Mac, iPad, tablet, Kindle, etc. Please help guide other potential buyers by writing a thoughtful review of it in Spanish – much appreciated!
And the print version (in English) will be ready in a few weeks, and available from Amazon, so if you prefer a paper book, stand by!