All else equal, would you invest in a company whose drug candidate reduces the risk of stroke by 1% or 50%? You probably chose 50%.
But what if the company whose drug reduces the risk of stroke by 50% actually reduces the risk of stroke by 1%? And what if the company didn’t lie about that 50% risk reduction either?
Impossible? Not really. It’s a statistical trick used by healthcare companies in their press releases and presentations in order to make things sound better than they really are.
And therein lies the problem. This is a truthful, but potentially very misleading, way to easily swing investor (or patient) confidence in a company’s or drug’s favor.
So what exactly is going on here?
To showcase this trick and what you should be aware of, let’s create a fictitious example. Any similarities to any real company/drug are purely accidental.
Let’s say that BioFarma Inc. has completed a clinical trial of their drug candidate, Trubador. The results are in. The company trumpets them in a press release and a presentation. Trubador reduces the risk of stroke by 50% when compared to a control. The media is all over it.
A 50% drop in the risk of something as deadly as stroke is, indeed, extremely impressive and that alone will certainly lure in many investors, clinicians, and patients.
Should it though?
RELATIVE RISK REDUCTION
Without any information to the contrary, you can bet that most of the time amazing results such as these are based on something known as relative risk reduction (RRR).
In this case, the RRR is 50%. This is a proportional reduction in the risk of a harmful event (stroke) between those in the Trubador group and those in the control group.
To illustrate this concept, let’s say BioFarma’s trial had 100 people in the treatment group and 100 people in the control group. 1% (from 1/100) of people in the treatment group had a stroke and 2% of people (from 2/100) in the control group had a stroke.
1% is half of 2% and so this is a proportional (a relative) risk reduction of 50%.
ABSOLUTE RISK REDUCTION
The problem is that this number is only half of the story with respect to risk reduction. The other half is often nowhere near as impressive.
This “least impressive” bit is known as the absolute risk reduction (ARR).
This is the straight up intrinsic reduction in risk. In other words, the reduction in risk between Trubador and the control group that isn’t qualified by potentially misleading things like proportions.
In this example, Trubador reduces the risk of stroke intrinsically by only 1% when compared to the placebo (from 2% in the Trubador group minus 1% in the control group).
1%. Impressive? Not nearly as much as that 50% RRR.
You can now appreciate why BioFarma wouldn’t tout the fact that Trubador decreases the risk of stroke by a measly 1% and would, instead, reveal an apparent 50% decrease in the chances of having a stroke while on Trubador.
WHAT THE FDA CONSIDERS WITH RESPECT TO THE ARR AND RRR
But you better believe that the FDA will be looking at the more conservative ARR and making decisions based on that number as well.
Of course, the FDA doesn’t approve or reject a drug purely on a single number such as this. Instead it will compare other data to the RRR and ARR in order to come to a decision.
One of the things the FDA does is a risk/benefit analysis.
Let’s just say that the ARR (the conservative number) is truly an amazing 50% for Trubador. This, on its own, would appear to exponentially increase the chances of Trubador’s approval.
Alas, that’s not enough. The FDA will look at Trubador’s safety profile, not just its efficacy.
In this instance, if the FDA also finds that the chance of having a life-threatening side effect while using Trubador is 95%, then that 50% reduction in stroke doesn’t look so great anymore.
On the other hand, let’s just say that the ARR is truly 1%. Is this grounds for rejection?
Absolutely not. The FDA will also look at things like need. It could be that a drug has a small ARR but it’s pretty safe, intended to treat a rare disease for which no other options exist, and so any improvement is worthwhile.
In the end, the takeaway here is simple: don’t judge a number by its cover and don’t fall for a statistical trap designed to sway you with only half of the truth.
As you would try and figure out the true relevance and implication of a number from a financial statement, so too should you do so for clinical trial results in order to better gauge the chances of FDA approval or even the outcome of a future clinical trial.
Last Updated: 2/28/2018
Author: Artem Cheprasov