A conversation with Dr Jess Dart, Jamie Gamble and Kate McKegg
What happens when suddenly everything changes and the old ways of working just…don’t?
Creators of our brand new course on developmental evaluation met virtually to discuss how developmental evaluation can help evaluators and social innovators respond and adapt to ever-more complex and changeable environments. But first – what exactly is developmental evaluation? And what is it not?
Can you give us some examples of what developmental evaluation is not?
It’s not about abandoning evaluation – it’s about bringing the best of evaluation into a social initiative, including critical evaluative thinking, use of evidence, use of data.
Developmental evaluation is not just about tracking and monitoring and describing what’s going on, nor is it about developing long term evaluation plans and designs, although it’s likely to use monitoring data to inform inquiry, questioning, and evaluative thinking as an initiative is unfolding. Developmental Evaluation is likely to unfold in smaller cycles of design and implementation, and always has to be open to adapting when things change.
Dr Jess Dart
It’s not ‘evaluation light’. We still have to apply all the tools in our evaluation toolkit, but we have to do it on our feet. It’s agile evaluation – I’ve even sometimes seen it referred to as ‘realtime’ evaluation. We work alongside program design – we bring evaluation tools to the table, and we have rigour around testing things.
Why is this form of evaluation important, and why you think it’s needed?
I love how much more attuned DE can be to the immediate learning and inquiry needs of people innovating and engaging in really tricky and tough issues. I also love how it can work with the rhythm of those on the ground, with our communities, and not be squashed into a timeframe or design frame that is imposed by someone else. It really can be by us, for us.
I think we’ve reached a saturation point for traditional problem solving. We’ve actually made good progress on technical problems, but we’re facing diminishing returns on deploying those traditional approaches to the significant problem areas that remain, for example poverty, climate, inequality. We need tools approaches and resources that support the ability to scale innovation and to adapt as needed to find our way through these massive challenges we face. Developmental evaluation is optimally suited to do that.
Dr Jess Dart
The time for social innovation is now. That means learning your way through change, trying things you haven’t tried before and working with communities in new ways. As evaluators and social change makers, we can see that these issues are getting harder to address, so how does evaluation sit with that?
Developmental evaluation can power new ways of working – it can help make initiatives better, scale faster, achieve greater impact, and even “de-risk” what you’re doing. Some governmental will only support social innovation if it is accompanied by evaluation, because it de-risks the design process. For example, if you’re looking at how well the engagement is going, if you’re looking at how people are responding, how safe it is – if you’re evaluating your initiative or program from all angles at the same time, then it’s a safer ride.
It’s already proving a really appropriate methodology for all the changes people have had to make throughout the pandemic.
What would you say to someone who is just about to start in a developmental evaluation role – what should they be looking out for?
Check that it’s really a developmental situation – is something table that involves serious innovation and complexity? Developmental evaluation tends to be a little more resource intensive, so you want to ensure you’re directing resources to where they can achieve the most.
Secondly, are you – your organisation, your initiative – really ready to learn? Do you have the authority, the will, the capacity to adapt? Because while developmental evaluation may help you may learn a ton about how your project is creating impact (or not), if you’re in a rigid system and you can’t actually apply what’s being learned then you can short-change the development. Check readiness before you proceed.
It’s called evaluation for a reason – it’s got “value” in the middle. While developmental evaluation is not evidence light, it is values rich. We need a critical practice in developmental evaluation to discover, explore and unpack our assumptions and the values that we hold and that others hold, because they make a difference to the choices that we make, they impact decisions we make about how we proceed, and those will affect someone.
It’s important to get to the heart of the things that are really important to you, your community and everyone with a stake in your project – what really matters and what you are trying to achieve? What counts as valuable? What would you see, hear, and feel and what would the facts be if you saw value in this initiative, and asking these questions often.
Dr Jess Dart
I’d say don’t rush into evaluation planning, or spend a large part of your time developing a big evaluation plan upfront. Try to be useful and stand behind the social innovation process, and let the evaluation plan unfold, perhaps later than you usually would, and try to keep it light, as it’s likely to change. In fact, it has to change as the thing you’re developing starts to develop, because you don’t yet know exactly what you’re going to be doing. Don’t use up your resources developing up detailed plans at the outset – it’s better to be useful.