What evidence should be used to make decisions about health interventions?

When making decisions about health interventions in whole populations, many people believe that the best evidence comes from analysis of the results of randomized control trials (RCTs). This belief is reinforced by the notion of a hierarchy of evidence in which the RCT is close to the pinnacle of evidence. It has that position because the RCT is a powerful tool for eliminating bias. Through randomization and careful following of protocols, the likelihood that an observed effect in a study group compared to a control group is due solely to the intervention rather than some other factor is greatly increased.

My colleagues and I agree that the RCT is the best form of evidence to use when making decisions about new drugs and technologies. However, there are often circumstances when a robust scientific decision can be made about the benefit of an intervention without the use of RCTs. This is very important when considering public health interventions because there are few RCTs of public health interventions on which to base decisions.

The crux of this idea is that you need a RCT only when you are in doubt that an intervention will produce a net benefit. Consider, for example, a new drug that has been tested and considered safe to trial in humans. You can’t know beyond reasonable doubt that this new drug does not cause harm so you need a RCT. The first function of a RCT is to establish with statistical confidence that an intervention is effective i.e. causes benefit rather than harm. Once an RCT has shown that the study group did better than the control group the issue is no longer whether the intervention is effective (causes benefit rather than harm) but whether the intervention is more cost effective than alternatives.

Now, consider a public health intervention that is highly unlikely to cause harm, for example, one that is about getting one group to have the health behaviour of another group in the population. Such an intervention would be giving families that do not eat many vegetables subsidized vegetables and cooking classes in an attempt to increase their vegetable consumption. We would argue that for this intervention you don’t need RCT evidence because you know beyond reasonable doubt that the intervention does not cause net harm. You know that increased vegetable consumption will improve health, albeit by a tiny amount, and you also know it would be perverse if making vegetables cheaper and inspiring people to use them made their overall diet poorer. Before making a decision about whether to implement the intervention, you do, however, need to consider its cost effectiveness. Are the benefits (increased vegetable consumption) worth the cost (subsidy and cooking class), and could the resource be better spent on an alternative?

Current practice for recommending health interventions tends to treat all interventions as if they were harmful and seeks definitive evidence of effectiveness. When trials do not exist, current practice often declares a lack of conclusive evidence of effectiveness and this acts as a huge barrier to implementation. This means that many cheap potentially cost effective interventions are disregarded not because they don’t work, but because there are no trials able to statistically demonstrate their benefit. What is missing in this process is the correct use of theory. In many cases where there is an absence of good trials, theory and causal models can be used to establish beyond reasonable doubt that a public health intervention does not cause harm. Our proposed approach is to consider all relevant evidence, including theory, in a transparent process that aims to reduce doubt about whether an intervention causes net harm. If we do this we can uncover many harmless cheap interventions which have important benefits at the whole population level and implement those that are found to be cost effective.

Heading image: Test tubes by PublicDomainImages. CC0 via Pixabay.