by Kate Nichols, Senior Consultant

• It was timely to learn more about the application of randomised control trials (RCTs), including where they might be appropriate and compelling. At the same time, learning more about RCTs reinforced the value and place of the type of work we do at Clear Horizon which tends not to include experimental design
• I found Bill Trochim’s concept of pieces of evidence (i.e. pieces of evidence used in a court room) versus a synthetic body of evidence really useful. This has really change my view of what evidence and evidence-based decision or policy making is really about
• At Clear Horizon we work a lot with peoples’ anecdotes and/or narratives. I think there’s scope for us to strengthen this methodological preference by triangulating interview data with secondary data in a more systematic way
• There was a lot of discussion around systematic reviews, especially in the context of evidence based decision making. Given the relatively standardised approach to the performance story reports (PSRs) we’ve been undertaking for the Australian Government (and others), i think there’s opportunity for us to explore the application of such approaches in relation to these pieces of work
• PSRs could be really useful for large organisations such as CSIRO who are attempting to conduct systematic evaluation across their component businesses as well as flagship projects, which are like a large slice of investment. PSRs would be particularly useful for examining the impact of the flagship slices
• DPI is doing some really interesting experimentation in the area of cost-benefit analysis - it’s an interesting pasture and i like the approach they’re taking (i.e. a collaborative, multi disciplinary team approach). For a methodology that results in findings that some people invest a lot of faith in, I was struck by the large number of assumptions that underpin the approach. As part of their work, DPI are also tackling the counterfactual. I’d love to learn more about how DPI are approaching it, particularly given the importance of DPI in our broader suite of work.