The what else tool: A basic way to strengthen your impact claims and avoid having egg on your face! By Jess Dart
Posted on 03 Aug 2018 under Blog Post & Evaluation Resources
I
have been getting excited about process
tracing recently; for example, as used in this
report that evaluates 25 collective impact initiatives. Process tracing is
a way to back a claim about contribution without using experimental techniques.
However, when I talk about process tracing to my non-evaluator colleagues,
their eyes glaze over and I can almost hear them groan. They want to know: “Isn’t
there a simpler approach we can use ourselves?” So while the geek in me loves
process tracing and contribution analysis (and I do recommend this to
evaluators), I can really see the need for a more basic approach to making a
contribution claim.
Recently I came across a program team all excited because they had seen a lift
in school attainment rates in the school they were working in. They came back
and told their client and lots of their colleagues. They had a bit of egg on
their faces when they discovered that all the schools in the district showed the
same increase – including the ones they weren’t working in. This is a classic
case of over-claiming and not doing the due diligence to check up on your
impact claim. It’s scenarios like this that make me think all projects should
at least do the “bare basics” when making an impact claim. To meet this need, I
have put together a tool. I have called it the “what else test” – the idea
being to think about what else might have caused the result other than you – or
even what else caused it as well as you.