A pivotal trial failure, however spectacular, rarely causes discontinuation of a clinical project. This is biotech, after all: there is serious post hoc data crunching to be done in an effort to convince the markets that the result might have been positive.
A case in point is Immupharma, whose lead project, Lupuzor, failed its pivotal trial in lupus in April; the result of the intervening six weeks’ data interrogation was revealed today, and victory was claimed in the subgroup of a subgroup. Though the group’s UK stock was up 7% today, investors should think hard before giving the data credence (see table below).
After all, today’s claim of statistical significance favouring Lupuzor does not even relate to a prespecified subgroup – which in any case would have been spurious, given the failure of the primary endpoint. It relates to the post hoc analysis of a subgroup of patients within another post hoc subgroup.
This sub-subgroup comprises subjects who had anti-dsDNA autoantibodies – these correlate with them actually having lupus, Immupharma states – as well as having been treated in hospitals in Europe and Mauritius combined, and not at US centres. The primary endpoint, response in all-comers, was of course a clear failure (Lupuzor reminds UK investors that there’s no easy money in biotech, April 17, 2018).
How the company got to this apparently positive cohort is not entirely atypical of biotech groups that reanalyse failed data for hints of a path forward. Already in April Immupharma was suggesting that it was seeing activity in subjects who actually completed the study, and separately in those who had anti-dsDNA autoantibodies, though it could not claim a p value below 0.05 for either analysis.
For it to find a group that yielded a nominal p value below the magic 0.05 it had to carry out a further data cut, and the geographical split proved to be just the ticket.
|How Immupharma found its sub-subgroup for Lupuzor|
|Lupuzor||Control (placebo+SoC)||p value|
|All comers n||101||101|
|All comers responding*||53 (52.5%)||45 (44.6%)||0.2631|
|Completers responding||53 (68.8%)||45 (59.2%)||0.2408**|
|Europe + Mauritius n||65||65|
|Europe + Mauritius responding||not disclosed||not disclosed||?|
|Anti-dsDNA Ab+ve n||52||55|
|Anti-dsDNA Ab+ve responding||32 (61.5%)||26 (47.3%)||0.967|
|Eur + Maur + anti dsDNA n||38||41|
|Eur + Maur + anti dsDNA responding||27 (71.1%)||20 (48.8%)||0.0218|
|Notes: *primary endpoint; **completer numbers extrapolated from Immupharma's statement, and p value generated via a Fisher's exact test calculator. Trial ID: NCT02504645.|
What is highly unusual here is Immupharma then saying that the nominal p<0.05 that this exploratory analysis yielded was “statistically significant”, as today’s statement claims.
Typically, multiple data analyses require a statistical penalty to be taken each time a dataset is interrogated, meaning that with each analysis the p value that has to be beaten becomes tougher. Moreover, in a trial as small as this, most of the powering will have been assigned to the primary analysis, which failed, leaving barely any “alpha” for subsequent endpoints.
Immupharma told EP Vantage that the statistical threshold it assumed for its sub-subgroup analysis was 0.05, adding: “No penalty was taken.” Investors should bear this in mind if Immupharma subsequently decides to seek additional funding from the market on the back of these data.
There are other inconsistencies, too. For instance, the result from the 11 US centres that participated in the pivotal study directly contradicts data from Europe and Mauritius, and shows patients with anti-dsDNA autoantibodies on placebo responding better than those given Lupuzor.
Perhaps the most glaring of all is the result in all-comers who did not have anti-dsDNA autoantibodies. True, the numbers are small, but the response among placebo recipients here was a staggering 65%.
Without a watertight explanation such paradoxical data call the entire study into question. Immupharma offered no explanation for the poor performance of US patients, saying only that this could have been “due to a number of differing factors between the two cohorts, which requires further investigation”.
UK investors do not need to have particularly long memories to recall Lupuzor’s disastrous development track record.
At the very best the pivotal data dredge points to Immupharma having to do another phase III trial, recruiting only subjects who correspond to the sub-subgroup it claims to have identified. That, as everyone knows, will require more money.