Incomplete reporting of COVID-19 disease severity criteria compromises meta-analysis

The World Health Organization’s (WHO) guidelines for the treatment of COVID-19 are based on the combined analyses of published randomised control trials (RCTs), using statistical methods that allow for the comparison of various treatments between studies.

The authors of Definitions matter: heterogeneity of COVID-19 disease severity criteria and incomplete reporting compromise meta-analysis reviewed the 81 studies included in the WHO COVID-19 analysis and compared their severity classifications with those used by another international COVID network. The two were the same in only 35% of trials. Of the RCTs evaluated, 69% were considered by the WHO group to include patients with a range of severities. The distribution of disease severities within these groups usually could not be determined, and data on the duration of illness and/or oxygen saturation values were often missing.

This imprecision in severity assessment compromises the validity of some therapeutic recommendations. Crucially, it shows that the extrapolation of “lack of therapeutic benefit” shown in hospitalised severely-ill patients on respiratory support to ambulant, mildly-ill patients should not be done. To address these issues, the researchers recommend using individual patient data (IPD) to guide and improve therapeutic recommendations for COVID-19.

Lead author, Professor Philippe Guérin, Director of the Infectious Diseases Data Observatory (IDDO) said: “The slow global rollout of vaccines and the threat of new variants means that effective therapeutics are still urgently needed, and information on COVID-19 disease severity in clinical trials is critical for the interpretation of therapeutic responses.

 “Care must be taken not to over-interpret currently available data and substantial uncertainty remains for the treatment of patients with early COVID-19. Individual patient data meta-analyses (IPD) are needed now to guide and improve therapeutic recommendations and these require international collaborations following FAIR principles to ensure equitable data sharing. IPD meta-analyses are key to assessing drug efficacy in the large majority of patients who do not yet require hospitalisation.”

The published literature contains a wide variety of COVID-19 severity threshold criteria, definitions and categories. Many of these are arbitrary and to add to the confusion, during the course of the pandemic some of the definitions were changed. Often, the key measures of severity were not reported at all and when they were, they were frequently incomplete or ambiguous.

The majority of the randomised controlled trials (RCTs) which formed the evidence base for the WHO therapeutic guidelines included mixed populations, with unclear distributions of disease severity and associated outcomes, and significant variability in observed mortalities even among groups classified as having similar severities. This is a significant concern because in COVID-19 there is strong evidence for heterogeneity of treatment effects according to the stage of disease at the time of treatment.

While the large RCTs in hospitalised patients have provided robust evidence and guidance for the management of severe COVID-19, substantial uncertainty remains for chemoprevention and the treatment of early COVID-19. Epidemiological and therapeutic assessments would benefit substantially from agreement on definitions of severity and the full reporting of key clinical measures, so that the quality and value of meta-analyses can be improved.

Read the paper

About IDDO and COVID-19

The Infectious Diseases Data Observatory (IDDO) is a scientifically independent, multi-disciplinary coalition of the global infectious disease community. It provides the methods, governance and infrastructure to translate data into evidence that improves outcomes for patients worldwide. IDDO hosts one of the largest international collections of clinical data related to COVID-19 with detailed individual patient data on more than 700,000 hospitalised individuals from over 1,200 institutions and over 45 countries. IDDO has facilitated the re-use of these data in 48 novel analyses to date.

IDDO builds on and incorporates the pioneering work of the WorldWide Antimalarial Resistance Network (WWARN), a collaborative data-sharing framework begun in 2009 that proved it was possible to produce policy-changing scientific evidence from historical data.