The 9 most common Measurement, Evaluation and Learning questions – answered!
If you’re in the business of creating social or environmental change, you might have a sinking feeling when you think about impact measurement. Data? Evidence? Indicators?
There’s also a good chance you’ve come across the terms “Monitoring and Evaluation”, “Impact Measurement” and “MEL” and gotten lost in the terminology. In fact, there’s a lot of terminology associated with the world of change measurement, let alone the number of approaches and theories on how best to do it. Which is why it’s sometimes a project planning afterthought (if thought about at all). And it shouldn’t be.
Properly planned for and used, your impact measurement system and the thinking that comes with it can be one of your most powerful tools for driving and amplifying change.
These are the top questions our learners ask when they are figuring out how and where to start
1. What is Measurement, Evaluation and Learning (MEL)? (and why should I care?)
Starting with the basics – MEL stands for Measurement, Evaluation and Learning. It’s the structured approach we use to determine whether the time, money, and efforts we’re dedicating to a project or initiative are making a real difference. Getting a handle on the type, quality, and quantity of impact we’re achieving helps us figure out what’s working – and, even more importantly – what’s not.
Using the structured approach to measuring their impact helps a program or initiative focus resources in the areas that are achieving the greatest change – in other words, more bang for buck. Likewise, it can help you identify and adapt the activities and approaches in initiatives that are not achieving the desired outcome – before you’ve invested too much.
Funders are increasingly wanting to see demonstrable evidence of impact and change made, as opposed to reporting on activities and how much was spent. It means that any changemaker needs to consider what type of systems and thinking is required to show the impact they are making. A structured approach like MEL incorporated into the initiative is key.
2. What’s the difference between Measurement and Evaluation?
Measurement is all about collecting good data (both through numbers and stories) consistently throughout the project life cycle. This can include details of attendance at events (who and how many), event feedback forms, meeting minutes or monthly project reports. By collecting good measurement data throughout the project cycle, small adjustments can be made to the project based on what the stories and numbers tell us.
Evaluation is using this measurement data and collecting additional data (both quantitative and qualitative) to understand how the project is going at a point in time. Your measurement will tell you that 20 people attended your event. Evaluation helps you make a value judgement about what the measurement tells you – e.g., whether getting 20 attendees is positive or negative. Evaluation helps you determine the merit or worth of the project (how is it going overall?).
3. Gotcha. What is the “L” in MEL then?
(This is our favourite!) The L in MEL stands for Learning. This is where the greatest value of MEL lies, but it’s also the most commonly overlooked part. Learning in practice is deliberately dedicating time and building in regular processes to make sense of what the data is telling us and adapting our program accordingly. It’s the part that makes all that measurement and evaluation work useful. After all, what’s the point of finding things out if we don’t use them to drive change – or waiting until the end of a project to see if it worked?
This learning doesn’t happen by osmosis – we must plan for it. This means building in structures for learning as part of your overall project or program management. When you’re starting out, this can be as simple as scheduling a regular reflection workshop.
4. So where do I start?
MEL is a great tool for clarifying the change you are trying to make. After all, you need to be clear on the types of change you want or are expecting to see before attempting to measure or evaluate it.
Depending on the area of change or sector you work in, you might be familiar with a Theory of Change, Program Logic or a Program Theory. This is the roadmap for how you think you will arrive at your change destination. When starting your MEL journey, it’s a good idea to revisit (or even recreate!) your Theory of Change, testing your assumptions and checking that everyone working on your initiative has a shared understanding of how you will collectively create change.
Once you have clarified your Theory of Change, you can then find ways to measure the changes in a systematic and easy way.
5. This seems like a lot of work and I’m very busy. How can I make it quick?
In short? Good planning and integration. In the long run, MEL can save you time – you’ll gain program efficiencies by spending less time and resources on activities that aren’t as effective. (After all, do you want to be putting in long hours for something you can’t be sure is actually working?)
But you will need to invest in setting up your MEL framework and getting everyone on board for your shared journey of change. And, as per question 3, you need to build in the time learn, which we guarantee will deliver program efficiencies.
Once you’ve got the basics covered and people on board, MEL doesn’t have to be another thing to do “on top” of your existing work. There are effective ways to weave MEL practices and evaluative thinking into your existing work and processes. For example, working out where MEL data gathering can be integrated with existing data and survey work.
6. But what if MEL shows my project isn’t working?
We get this question a lot. Sometimes MEL can feel like a test or audit of our work (and of us!), and as such, people can tend to avoid it. (The term “evaluation police” may have come up once or twice). We’ve already touched on some of the best ways to combat this: building a shared understanding of your journey of change, a common approach for measuring your impact, and most importantly, supporting a culture of learning.
And sometimes the data will show that a project isn’t working. But if you genuinely want to create change, then wouldn’t you prefer to know that sooner rather than later? And be armed with the best data possible to understand why something’s not working, so you can pivot if need be? Everyone makes mistakes, and things sometimes don’t work as intended. Good MEL is about learning from mistakes – helping you identify problems quickly and providing the space and structures to adapt and improve
7. My role’s not MEL – how does this affect me?
Are you involved in making project decisions? Do you need to understand what’s working well and what needs tweaking? Have you ever been asked to collect data and never understood why? Then MEL affects you.
Even if you’re not responsible for driving or setting up MEL in your organisation, you need to understand how your role or team is contributing to the change the initiative is seeking to achieve. You might be able to contribute to better understanding telling the story, for example, through data collection, and how you can use the information collected from the MEL system to learn about what is working well or what may need to be improved.
8. How much should I budget for MEL?
A general rule of thumb is between 10-20% of your total program spend should be dedicated to MEL. And while initially this might cause some organisations to balk, the reality is they’ll then be spending 80-90% of their program spend on something that can be systematically measured, evaluated and improved. As opposed to having 100% of program funds spent on something that may or may not be working. We know which we’d rather.
9. I’m sold – how do I convince others in my organisation that MEL is important?
The first step is to understand what’s important to your stakeholders – what are they trying to achieve and why? Think about how MEL might help them achieve their goals and then build your case, drawing on real-world evidence. This might include case studies demonstrating the benefits of MEL in similar organisations or initiatives.
Better still, develop and implement a MEL plan for a single program or initiative in your organisation as an internal trial. The idea is to show that you can create great Involve your internal stakeholders throughout the process so they can see the benefits first-hand.
