Developing a mixed methods framework for process evaluations of complex interventions: the case of the National Exercise Referral Scheme Policy Trial in Wales

TL;DR: A mixed-method framework exploring programme theory, diffusion, implementation, participant experiences and reach is applied to the evaluation of the Welsh National Exercise Referral Scheme (NERS), though some divergence from programme theory emerged.

Abstract: Where possible, policies to improve public health should be evidence-based. Where political pressures and shortage of evidence force action in advance of evidence, effectiveness can be evaluated during policy rollout. Because the aetiology of public health issues is complex, successful policies will likely be complex in their design, their implementation and their interaction with their contexts and target audiences. Process evaluation is therefore crucial in order to inform consistent implementation, and alongside outcomes evaluation, in order to understand how outcomes are produced. However, limited methodological guidance exists for process evaluation. This thesis develops a mixed-method framework exploring programme theory, diffusion, implementation, participant experiences and reach, which is applied to the evaluation of the Welsh National Exercise Referral Scheme (NERS). A logic model is developed via discussions with policy representatives. Diffusion is explored via qualitative interviews with policy representatives and local coordinators. Implementation checks draw on routine data, observation and self-report. Participant experiences are explored via qualitative interviews. Social patterning in reach is explored using routine monitoring data. The study identifies challenges diffusing NERS into local practice, in relation to communication structures, support, training provision and the mutual adaptation of the scheme and its contexts. Implementation checks indicate a common core of discounted, supervised, group-based exercise, though some divergence from programme theory emerged, with unfamiliar activities such as motivational interviewing and patient follow-up protocols delivered poorly. Nevertheless, relatively high adherence rates were achieved. Key perceived active ingredients in practice included professional supervision, enabling patients to build confidence and learn to exercise safely, and the patient-only environment, seen as providing an empathic context and realistic role models. However, lower uptake emerged amongst non-car owners, with higher adherence amongst patients already moderately active at baseline, older patients and non-mental health patients. Implications for ERS implementation, outcomes interpretation and process evaluation methodology are discussed.