Current approach to exclude indeterminate recurrence from research analysis leads to an overestimate of the efficacy of antimalarial drugs

It is the continuation of two earlier studies which explored approaches for comparing different statistical methods for deriving cumulative estimates of drug efficacy from clinical studies.

This latest paper looked at the tools used to estimate the efficacy of antimalarial treatments. In antimalarial clinical efficacy studies for uncomplicated Plasmodium falciparum malaria, researchers often face situations where they cannot discriminate between the two different causes of recurrence; recrudescence – the same parasite causing the second episode of the disease, or a new infection. Under current WHO guidelines – the Complete Case (CC) method – these cases are defined as indeterminate outcomes and excluded from the analysis.

The study compared the estimates derived from the CC method with two other methods – the multiple imputation (MI) and inverse probability weighting (IPW) which allowed for the missing data to be included in the analysis. MI involves substituting the missing value multiple times using the observed covariates while IPW weights the complete observations based on their estimated probability of missingness.

The authors set out to test what would happen if indeterminate outcomes were included in the analysis, comparing the different approaches of considering them as missing data, and comparing results to the current WHO recommendation.

Professor Umberto Dalessandro, Director of the MRC Unit The Gambia at the London School of Hygiene and Tropical Medicine, said: "The results of this secondary analysis on a large trial we carried out a few years ago are extremely useful and important. The estimation of the efficacy of antimalarial treatments requires discriminating, among recurrent infections identified during the patients' follow up, infections that have not been cleared by the treatment and those newly established during the follow up. This is done by genotyping them. In some cases, such analysis is unable to provide an answer, the infections are indeterminate. The proposed approaches provide a way of handling this issue and of estimating with higher precision the efficacy of antimalarial treatments."

Dr Prabin Dahal, first author on the study, said: "We argue that we should include indeterminate outcomes in the analysis because they can be considered as missing data which allows us to use the well-established and statistically principled approaches."

The researchers used a study originally involving more than 4000 individual patients to compare the performance of CC, MI and IPW approaches in estimating the Kaplan-Meier (K-M) probability of parasitic recrudescence. Based on results, the CC analysis resulted in an over estimation of the efficacy of antimalarials tested while MI and IPW provided a more accurate (consistent) estimation. The paper recommends that researchers should consider using either the MI or IPW for handling indeterminate outcomes when reporting results of antimalarial trials.

Dr Dahal said: "The data we used for the simulations involved 4000 individuals – there was a lot of recrudescence, reinfections and a lot of indeterminate outcomes, so it was quite a powerful data tool to use to test this method. In one scenario where more than 30% recurrences were indeterminate, we found that this can overestimate the efficacy estimate by 2-3%. This can have implications for the introduction of new drugs as a first-line therapy, as currently new treatments must demonstrate an efficacy of >95%.

"The IPW and MI approaches are easily implemented in standard statistical software and should be considered for handling indeterminate outcomes, at least as a sensitivity analysis, in the derivation of antimalarial failure. We suggest that caution should be exercised in small sample trials with relatively few recurrences. However, we strongly suggest that these approaches should be routinely used when carrying out individual participant data meta-analysis, where the application of MI or IPW is much more feasible."

Dr Mateusz Plucinski, an epidemiologist with the US Centers for Disease Control and Prevention's Division of Parasitic Diseases and Malaria, said: "It is important that investigators performing antimalarial therapeutic efficacy surveillance use the most unbiased and robust statistical analyses possible.

"Underestimation of treatment failure rates can lead to a failing drug continuing to be used, resulting in increased and avoidable morbidity and mortality. The proposed approaches for analysis of indeterminate genotyping results advance best practices for analysis of therapeutic efficacy data."

Read the research: Dahal et al Dealing with indeterminate outcomes in antimalarial drug efficacy trials: a comparison between complete case analysis, multiple imputation and inverse probability weighting. BMC Research Methodology. Published: November 28, 2019.

This work was done as part of the WWARN Clinical Scientific Group.