Recently we have noticed a groundswell of interest in how to evaluate policy influence; a notoriously slippery thing to catch hold of. This is in part due to the multitude of forces that drive policy change but also because change usually happens over a long period with many different actors and interests. In addition, external ‘environmental’ factors such as unforeseen events often change the agenda, creating opportunities and risks for policy makers.

How do we untangle this morass? This is a challenge that evaluators, bureaucrats and social scientists have been grappling with for some time.

There is an emerging body of knowledge on this topic that has come primarily out of the United Kingdom through think tanks such as the Overseas Development Institute (ODI).

Last year at Clear Horizon we trialled a policy evaluation approach developed by ODI called an Episode Study as part of a global forestry research programme. We used a modified “light touch” version* of the Episode Study.

Episode studies take as their starting point a policy change and then work back from this change to trace the influence that a particular organisation or intervention made in bringing it about. An important part of the method is that it also looks at what other factors/actors influenced the change, while it also seeks to rule out alternative explanations. This has a parallel to a planning and evaluation tool we often use called people centred program logic, which similarly highlights the actions of people and organisations in bringing about change. This is of course shaped by the contexts and the environments (political and otherwise) that we work within.

In our “light touch” episode study, the change under investigation was the Government of Rwanda’s commitment to adopt the recommendations of a nationwide forest and land-use assessment conducted under the global Bonn Challenge to undertake forest landscape restoration. Working back from this commitment we created a timeline of events that showed that together with the Government, the authors of the assessment - the International Union for the Conservation of Nature (IUCN) and the World Resources Institute (WRI) - had been influential in contributing towards the commitment.

We decided to frame the overall findings by drawing on another similar approach called Significant Policy Influence (SPI) that we have been using in Indonesia recently. The format we use in SPI for investigating policy change covers:

•                   A description of the outcome or policy change

•                   The significance of the outcome

•                   The relative contribution of the intervention or organisation

•                   Evidence to support the above

We found that this presentation format worked well with the Episode Study approach. Of course these approaches are just a few examples of policy change evaluation approaches.

A copy of the Episode Study report we prepared for the IUCN on the Rwandan Forest Landscape Restoration assessment can be found on the IUCN website here.

*”Light touch” is an expression we are using increasingly to refer to the use of an approach or method that is done with limited resources with a pragmatic focus on user needs and simplicity.