A multiple sclerosis study of Merck KGaA’s evobrutinib has provided investors with a valuable lesson in data analysis. Efficacy of the project’s two highest doses had previously been backed by statistically significant p values, but full results published today in the NEJM support only the middle dose, and show no numerical dose response.
At fault is that the dataset presented at last year’s Ectrims meeting had baldly cited unadjusted p values that did not correct for multiple analyses. Moreover, the NEJM paper dismissed Merck’s claim that evobrutinib had shown clinically relevant decreases in relapse rate, and highlighted an important adverse event.
It is of course vital to adjust p values for multiplicity, and investors should always ask whether this had been done before trusting a statistical analysis. Each time a dataset is interrogated the odds of a chance finding go up and, unless a correction is made, a result with a nominally positive p value could easily be a fluke.
Remarkably, such a correction had not been made when Merck first presented the evobrutinib phase II data at Ectrims last October.
The trial tested 25mg and 75mg once-daily doses, and a 75mg twice-daily dose, versus placebo in 267 relapsing MS subjects, and as primary endpoint compared total gadolinium-enhancing MRI lesions (an indication of MS relapse) at up to 24 weeks. 75mg once and twice daily were said to improve the lesion rate ratio, with p values of 0.0015 and 0.0313 respectively.
However, once adjusted for multiple comparisons the respective p values for these two groups rise to 0.005 and a non-significant 0.06, the NEJM paper reveals. The fact that the stats now only back a middle evobrutinib dose is conceptually problematic.
So is the fact that a key secondary measure, annualised relapse rate, had earlier been said to show “clinically relevant decreases”. Now the paper’s authors, led by Dr Xavier Montalbal of Vall d’Hebron Barcelona Hospital, state bluntly: “Evobrutinib at any dose had no effect on the annualised relapse rate or disability progression.”
And the authors add that the BTK inhibitor was associated with elevations in liver aminotransferase levels, an adverse event that should be seen as especially concerning now that efficacy looks questionable.
|Summary of evobrutinib's MS study (NCT02975349)|
|Measure||Placebo||25mg once daily||75mg once daily||75mg twice daily|
|Mean lesions at wk 12, 16, 20 & 24*||3.85||4.06||1.69||1.15|
|Lesion rate ratio vs placebo||NA||1.45||0.30||0.44|
|Unadjusted p value (per Ectrims)||NA||0.2950||0.0015||0.0313|
|Adjusted p value (per NEJM)||NA||0.32||0.005||0.06|
|Annualised relapse rate**||0.37||0.57||0.13||0.08|
|Source: NEJM. Note: *primary endpoint; **secondary endpoint.|
Merck had not responded to questions about these findings as Vantage went to press. However, it might be argued that in a relatively small trial like this any hint of efficacy, even in a single, middle dose, is positive.
The authors point to other limitations, such as that this study enrolled relatively old subjects with long-duration disease and relatively few pre-baseline relapses. Indeed, they suggest that some might have had secondary progressive rather than relapsing MS on entering the trial.
Still, the bar for an MS drug in the market is high, as competition would be not against placebo but against extremely efficacious treatments like Roche’s Ocrevus. The Merck study did include a patient cohort given Biogen’s Tecfidera, but no statistical analysis was carried out for it.
As for competition from other BTK inhibitor players, work in MS is very limited (German Merck boosts autoimmune BTK inhibitor chase, October 16, 2018). Beyond evobrutinib only one asset, Sanofi/Principia’s SAR442168, is in the clinic; its dose-finding trial is due to read out by the end of this year.
Investors should probably not expect a flurry of activity among developers of other BTK inhibitors just yet.