Recently, a few of the members from the Social Innovation Team at Clear Horizon were lucky enough to attend the Complexity and Evaluation conference, led by Collaboration for Impact. It was a busy couple of days and a fantastic chance to learn more about how to work in social innovation spaces in ways that don’t stifle big, messy and exciting, systems change initiatives.


Critical systems


Kate McKegg’s presentation on critical systems challenged us to critically reflect on our role within a system of change.

As evaluators, we have to consider our ethical responsibilities given the impact our actions and decisions can have on the communities the intervention may be trying to support. Engaging with critical systems means acknowledging that our role extends beyond passive observer and that we are implicated as agents of change within the system. There are many decision-making points we are often involved in, including: setting the Evaluand (including the boundaries of the system), clarifying the theory of change, choosing tools for monitoring and evaluating, making evaluative judgements, and deciding on who will be involved in the evaluation process.

Keeping in mind the practical implications of our actions raises a number of questions that we need to continually reflecting on. As Kate outlined, critical systems asks us to consider:

  • Motivation - where does our sense of purpose come from? How do we ensure this is driven by community change and not just intellectual curiosity or a desire to test shiny new models?
  • Power - who has, or should have, power over what is happening?
  • Knowledge - what expertise supports, or should support, decisions? What knowledge do we see as relevant and why?
  • Legitimacy - is the approach informed/led by those who are affected? Who is representing those that are affected but are not present for decision-making

So, what does this mean for our work as evaluation practitioners? Since the conference, our team have been reflecting on our practices and the changes we need to make within our organisation moving forward.

Involving lived experience in evaluation


The importance of community led change and the involvement of people with lived experience is spoken about regularly within the social change space yet are under-represented in positions that hold decision-making power, including as evaluators. The social change space is filled with skilled and dedicated people who are genuinely trying to influence positive social change, but it is important to regularly stop and consider why a particular approach is being taken and to what extent is it being driven by community need. Engaging with critical systems thinking reinforces our responsibility to ensure we the right people in the room so that community members with lived experience can be decision makers in their own community.

While Clear Horizon has a strong foundation in collaborative and capacity building approaches to evaluation, we are rethinking what we need to do to improve our practices in this area. We need to take a more active role to ensure we are meaningfully supporting the development of a space in which people from community or with lived experience are well-represented in positions of influence in the evaluation sector and in the social change space more generally. We are currently exploring the most effective role we can play, so watch this space.


Choosing criteria and methods for evaluation


Theory of change


We love Theory of Change here at Clear Horizon and believe it is an important tool to assist people to understand what their initiative is trying to achieve, test their assumptions and determine their M&E priorities. But as they say: every rose has its thorns, can’t fit a square peg in a round hole, etc. etc., you get the drift… Whilst working in the social innovation and systems space we are noticing that a clearly set out theory of change in complex and turbulent contexts can be difficult to establish and become redundant very quickly given the nature of interventions as being emergent and prone to change.

Evaluators need to adapt their approach to theory of change for systems change contexts.


Principles based evaluation


Social innovators are increasingly turning to principles based approach to guide the way they intervene in a system. Place-based and systems geared interventions acknowledge a particular intervention in one context might be entirely inappropriate for another, particularly when considering initiatives working within different cultural contexts. As approaches to evaluation in social innovation continue to evolve, we have to be flexible and adapt to continue to feedback relevant information.

Evaluators need to expand their tool sets for assessing appropriateness, including by engaging with principles based evaluation approaches.



Further Reading


Books 

  • Developmental Evaluation Examplars – Principles in Practice, edited by Michael Quinn Patton, Kate McKegg, and Nan Wehipeihana
  • Principles Focused Evaluation – The GUIDE, by Michael Quinn Patton

Other resources