aes19 International Evaluation Conference

Andrew Hawkins (ARTD  Consultants)

This paper is about identifying, managing and mitigating the risk that a program will not produce its intended effects. A principle of this approach is that a program at its core is simply a proposition that a certain course of action will lead to a certain set of outcomes. It is about putting the logic back in program logic.

Program Design Logic (PDL) is a tool for developing evidence based policy and programs. Through the language of 'necessary' and 'sufficient' conditions in place of 'outputs' and 'outcomes' it provides a framework to determine if a program or course of action makes sense 'on paper' before we attempt to determine if it makes sense in 'reality' through monitoring and evaluation.

The five types of risk are

1. It doesn't make sense on paper - logical risk
2. It makes sense on paper, but assumptions don't hold - assumption risk
3. It makes sense on paper, but we didn't do what we said we would do - performance risk
4. It makes sense on paper, assumptions hold, we do what we said we would do, but outputs don't materialise - theoretical risk
5. It makes sense on paper, assumptions hold, we do what we said we would do, outputs materialise, but intended outcomes don't follow, so the array of outputs was not actually sufficient to bring about a desired future state - logical risk
6. It makes sense on paper, assumptions hold, we do what we said we would do, outputs materialise, intended outcomes follow, but longer term outcomes don't materialise -external factor risk

This paper will discuss how a PDL approach can provide a comprehensive risk management framework before the first participant is even enrolled, which may then be managed and mitigated through program re-design as well as adaptive monitoring and evaluation.