aes19 International Evaluation Conference

James Smith (Menzies School of Health Research), Kalinda Griffiths (University of New South Wales), Moira Stronach (Northern Territory Department of Health), Liz Kasteel (Northern Territory Department of Health), Jenny Summerville (Northern Territory Primary Health Network), Julie Franzon (Northern Territory Primary Health Network), Michelle Ganzer (Northern Territory Department of Health), CCPMS Monitoring & Evaluation Working Group)

In 2010, the Northern Territory Government released a ten-year Chronic Conditions Prevention and Management Strategy (CCPMS). This was followed by the release of three separate implementation plans (2010-2012; 2014-2016; 2017-2020) across the CCPMS timeframe. A longer implementation timeframe was adopted to allow for the measurement of longer-term outcomes. The CCPMS and subsequent implementation plans clearly outlined guiding principles, key goals, key action areas, objectives, strategies and indicators/progress measures. In theory, the 'evaluation box was built and neatly wrapped' providing a useful platform to undertake monitoring and evaluation functions, which had been considered from the outset. However, it has recently surfaced that indicators/progress measures were poorly aligned to the objectives and strategies, and that in some instances data was not available to report against the indicators. Similarly, the indicators included in implementation plans changed across the life of the CCPMS, reflecting changes in policy direction and government priorities. This made it difficult to identify how best to measure the impact and outcomes of the CCPMS. That is, the 'structure of the evaluation box was weak'. In 2018, to address this conundrum, a multi-agency Monitoring and Evaluation Working Group, with independent Co-Chairs, was established to develop a retrospective Northern Territory CCPMS Monitoring and Evaluation Framework. In this presentation, we draw on multiple perspectives from the Working Group to track and discuss the process used to develop the framework. We will explain how we 'unwrapped, deconstructed and reconstructed the box'. We will explain how and why our multi-phased approach included: an indicator mapping process across multiple policy documents (2010-2018); preparing a retrospective logic model; identifying contemporary Indigenous evaluation principles; seeking expert advice on qualitative and quantitative measures; and prioritising indicators based on availability, utility or pre-existing reporting processes. In doing so, we will unpack the problems and possibilities encountered by the Working Group.