aes19 International Evaluation Conference

Brad Astbury (ARTD), Andrew Hawkins (ARTD)

Conducting an evaluation is never easy - it must be rigorous, practical, useful, real-world, and participatory. It is almost always the case that there is insufficient time to do it all in one study. But how do we determine what is the best approach right now? Rather than follow the latest fashion or treat everything as a nail because all we have is a hammer, we believe the best place to find answers is in the consolations of evaluation theory - that is, in examination and reflection on the fundamental questions that have occupied key evaluation thinkers over the last 60 years.

In this paper we present a series of ideas and conceptual maps that have been developed and used to focus any given evaluation. Each map or diagram considers similar fundamental issues and theorists but for slightly different uses.

The first speaker will articulate the nature and components of evaluation theory and distil insights from the 'big seven' theorists as identified in Shadish, Cook and Leviton's (1991) seminal text Foundations of Program Evaluation: Theories of Practice. A schematic of the practice-theory relationship in evaluation is offered to highlight ways in which various kinds of theory can be integrated to support and guide the design and conduct of evaluation.

The second speaker will present two maps- one on the 'information to cost ratio' that is designed for decisions about the most appropriate method for impact evaluation, and one on 'navigating uncertainty' which is broader and focused on the use of evaluation for piloting a path from a current problem to a desired future state. The second map integrates some newer theorists post 1991, particularly realist and complexity theorists.