Fifteen Eighty Four

Debates about the effectiveness of social interventions often begin with a deceptively simple question: ‘does it work?’ Yet this question obscures the deeper epistemic challenges at stake. Social policies and practices operate within richly detailed dynamic systems in which causal processes are rarely uniform or independent of context.  As a result, establishing whether an intervention is likely to work in a specific context requires more than observing outcomes elsewhere; it requires understanding how and under what conditions causal processes will unfold in the targeted context.

This book addresses that challenge directly. It develops a methodological framework for analysing causal processes in social systems and for assessing the warrant for claims about whether an intervention is likely to succeed – or has succeeded – in a particular context.

The Problem of Causal Diversity in Social Systems

Unlike physical phenomena, where causal regularities may often be stable across contexts, social interventions interact with heterogeneous conditions: organisational cultures, resource environments, professional norms, local histories and the behaviour of multiple agents. These contextual factors shape whether, and how, a causal process progresses.

Randomised controlled trials (RCTs), long regarded as the methodological gold standard for establishing ‘what works’, vividly illustrate this diversity. Variation in trial outcomes – both within individual studies and across replications – is not merely statistical noise. It reflects the fact that causal processes in social systems are contingent and context-sensitive.

Consequently, the central evaluative question cannot be restricted to ‘does this intervention work?’ but must instead be ‘will this intervention work here in this specific context with its rich panoply of possibly relevant features?’

Dealing with Diversity

The book proposes a systematic approach for identifying, examining and evaluating the context-specific causal processes that underpin social interventions. Using examples from many policy areas, including health and child protection, it demonstrates how to construct a rich context-local Theory of Change, going beyond the standard A à B à C to

• spell out what happens in the targeted context between A and B, and B and C
• identify what factors there help or hinder this process
• examine how these conditions can or do manifest in that context
• assess whether the causal process is capable of going through there (or post hoc, has succeeded in doing so). 

For instance, when a child protection agency adopts a new practice model, the framework encourages evaluators to ask whether there is empirical evidence of the model’s actual use in case records. Absence of such evidence is not a trivial detail; it signals a potential break in the causal chain.

Respecting Causation

A distinctive contribution of the book is its development of a new ‘theory of causation’ tailored to the complexities of social systems. Rather than privileging a single causal account, it synthesises seven widely accepted features of causation into an integrative framework. This allows evaluators to:

• draw on a broader and more diverse evidential base
• understand how different causal elements interact
• identify gaps in existing knowledge
• design effective implementations
• judge the strength of evidence supporting – or counting against – the claim that the intended causal process will (or did) occur in their context.

This integrative approach provides a more nuanced and defensible basis for causal inference in complex social settings.

Improving Methods

The field is increasingly recognising that the traditional effectiveness question is insufficient. Variation in outcomes across contexts demands a shift toward more discriminating evaluative questions: not ‘does it work’ but ‘will it work here’, ‘what are the context-local processes by which it could work here’ and ‘how should we implement it to make it more likely that these processes with go through?’ The methodology presented in this book offers a rigorous way to collect, organise and interpret evidence in order to answer these questions. It supports more robust, context-sensitive judgments about how policies and practices function in the real world.