aes19 International Evaluation Conference

Georgia Marett (ARTD Consultants), Jack Cassidy (ARTD Consultants)

This presentation shares insights into how behavioural economics and Behavioural Insights (BI) are used in program and service design and explores ways in which evaluation can and should take BI into account. A critical concept discussed in this paper is cognitive load. Research shows that cognitive overload can negatively impact decision-making and lead to more shallow processing of information and poor information retention. One method by which BI improves program decision-making and evaluation quality is by increasing the cognitive capacity of individuals.

We illustrate how service design can take cognitive load and BI into account and what might happen if BI are ignored when designing programs. Then, we examine and explain how to evaluate programs which have incorporated BI (including how cognitive load can be incorporated into a logic model, monitoring and evaluation frameworks and/or key evaluation questions). Finally, we conclude with a discussion about whether evaluation effectively uses the cognitive capacity of its stakeholders and practitioners.

This subject is important because while BI is a hot-topic in general and in evaluation, there is a lack of understanding about the ways in which it can be applied and how to evaluate those applications. Cognitive capacity is less well understood but is vital to understanding how to craft effective services, evaluate these services and conduct evaluations regardless of whether BI are included in the target of the evaluation. We will tie this into a realist perspective of evaluation through a discussion of how BI differ in effectiveness between people and situations.