What do we mean by strategic evaluation studies?

In addition to building a monitoring framework, programs and program teams may also need to think about including some evaluation studies that delve deeper than traditional monitoring. In addition to a deeper understanding of outcomes, evaluation studies can also include a review of the process (efficiency), the quality of the model (appropriateness), among other things.

Strategic: We call it “strategic” as we wish to emphasise that not everything needs evaluating, and that evaluation studies should be used strategically.

Evaluation: By evaluation we refer to a process that will lead to judgements about the worth or merit of a project or program. This may be focused on a number of different evaluation questions.

Study: By study we refer to that fact that some empirical evidence must be collected. Unlike monitoring, evaluation studies are often discrete (one-off) rather than on-going.

In order to plan for strategic evaluation studies we first need to think about what else we really need to know in addition to what the monitoring will provide.

Types of Evaluation

There are many different types of evaluations depending on the object being evaluated and the purpose of the evaluation. Perhaps the most important basic distinction in evaluation types is that between formative and summative evaluation.

Formative evaluations strengthen or improve the object being evaluated — they help form it by examining the delivery of the program or technology, the quality of its implementation, and the assessment of the organisational context, personnel, procedures, inputs, and so on.

Summative evaluations, in contrast, examine the effects or outcomes of some object — they summarize it by describing what happens subsequent to delivery of the program or technology; assessing whether the object can be said to have caused the outcome; determining the overall impact of the causal factor beyond only the immediate target outcomes; and, estimating the relative costs associated with the object.

Formative evaluation includes:

• needs assessment determines who needs the program, how great the need is, and what might work to meet the need
• clarification helps stakeholders define the program or technology, the target population, and the possible outcomes
• process evaluation investigates the process of delivering the program or technology, including alternative delivery procedures

Summative evaluation can also be subdivided:

• outcome evaluations investigate whether the program caused demonstrable effects on specifically defined target outcomes
• impact evaluation is broader and assesses the overall or net effects — intended or unintended — of the program as a whole
• cost-effectiveness and cost-benefit analysis address questions of efficiency by standardizing outcomes in terms of their costs and values
• secondary analysis reexamines existing data to address new questions or use methods not previously employed
• meta-analysis integrates the outcome estimates from multiple studies to arrive at an overall or summary judgement on an evaluation question