Natural experiments and health inequalities, what brings them together?

Natural experiments: why, when and what?

Much research is driven by the motto rerum cognoscere causas (to know the cause of things). There are several reasons why we can be interested in relationships of cause-effect. In my case, the main objective of causal evidence is to guide decision-making regarding our population health. This can be easily understood when thinking of the importance of randomized controlled trials (RCTs) that provide causal evidence on the effects of clinical interventions (e.g. drugs or surgical approaches) by design. In RCTs, the random allocation of individuals into the treatment, or control group, makes these groups equal in everything else but the treatment they receive. Hence, when comparing those groups, differences in results can be attributed to the treatment. However, RCTs are often unethical, unfeasible or simply impossible to conduct. This is particularly true for aspects that matter for inequalities in health among the population, as many of the social determinants of health (e.g. wealth or neighbourhood environment). Health care policies might also be challenging to study in trials, for example, changes in the costs paid by patients to use mental health services (cost-sharing).

Natural experiments present unique opportunities to investigate causal effects for many of these interventions or policies. Similarly to RCTs, we compare an intervention to a control group. However, these groups are not created by the researcher (as in trials) but by changes in the exposure caused by external shocks, or by factors outside researchers control (so-called exogenous variation). Typical examples of such shocks include: lotteries that increase some people´s wealth or unexpected weather disasters that make some people change neighbourhoods. Other sources of significant variation for health inequalities include the sequential roll-out of social programs or arbitrary thresholds imposed by policies or clinical guidelines. For example, suppose cost-sharing is implemented just for adults; in that case, we can study the impact of this policy by comparing individuals above 18 - that will have to pay for their health care -  with those just before turning 18 that are very similar but do not have to pay.

Embracing the challenges of Natural Experiments

Despite the potential of natural experiments to study meaningful cause-effect relationships, there are some challenges to conduct these studies. First, there are many interventions of interest for which the so-called exogenous variation will not exist, meaning that there is no control group that would be similar enough to the treatment so that we can compare them. Second, when there is exogenous variation, one of the big challenges can be unpredictability. In prospective evaluation of policies or programs, researchers may face difficulties aligning timelines of implementation, evaluation and funding. One can think of how legislations and reforms are sometimes implemented sooner than anticipated, while infrastructural interventions can be substantially delayed. I experienced that conducting evaluations retrospectively will not take challenges away: finding the data (which should have been collected in the past), accessing and linking these data, and meeting assumptions and statistical requirements often mean that many natural experiments scoped end up not being evaluated.

We can better tackle some of these challenges if we change current practices in research and policy-making. First, involving researchers in early phases of intervention/policy planning will avoid that the opportunities to evaluate those through natural experiments are missed. Second, research environments need to accommodate the uncertainty associated with conducting these studies. For example, quick and flexible funding sources could allow immediate data collection when legislation is suddenly implemented. When data already exists, multidisciplinary teams are essential to facilitate the timely linkage of databases within ethical and regulatory frameworks. This requires legal officers, policymakers, practitioners and researchers to collaborate. Last, training health professionals, non-academic stakeholders, funders, and policymakers in the value and specificities of natural experiments is key to increasing the confidence and use of these evaluation strategies. Only then stakeholders will recognize that natural experiments provide unique opportunities to strengthen the existing evidence, and that natural experiments are critical in the continuum of evidence from association to causality.

Natural experiments and health inequalities

In my PhD trajectory, I have been using natural experiments to study the effects of reforms and programs in mental and long-term care: the effects of increasing patient cost-sharing in access to mental health services for young adults, or the effects of receiving eligibility for long-term mental health care for mentally ill patients that cannot leave independently. These natural experiments contribute to evidence-based policy-making in health inequalities because they study differential effects of interventions by socioeconomic status (first example), or because the interventions target the most vulnerable groups (second example).

About the author

Francisca Vargas Lopes is a PhD candidate at the Department of Public Health at Erasmus MC with joint supervision from the Department of Applied Economics of Erasmus School of Economics. Working at the interface of these two disciplines has provided her with the tools to conduct evaluations of natural experiments in the domain of health care inequalities. This blog takes some of her views on these studies recently shared in a joint piece with Famke Molenberg, “The paradox of getting control over natural experiments”. You can contact Francisca via f.vargaslopes@erasmusmc.nl or learn more about her work here.