covid 19 case study

An official website of the United States government

The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• Browse Titles

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

National Academy of Medicine; The Learning Health System Series; Grossmann C, Chua PS, Ahmed M, et al., editors. Sharing Health Data: The Why, the Will, and the Way Forward. Washington (DC): National Academies Press (US); 2022.

Sharing Health Data: The Why, the Will, and the Way Forward.

• Hardcopy Version at National Academies Press

5 CASE STUDY: COVID-19 EVIDENCE ACCELERATOR

Interviewees: Carla Rodriguez-Watson, PhD, MPH, Director of Research, Reagan-Udall Foundation; and Jeff Allen, PhD, Executive Director, Friends of Cancer Research

The COVID-19 Evidence Accelerator (EA) is a research collaborative comprised of 230 public and private entities spanning the health care ecosystem. The project was launched in April 2020 at the behest of the U.S. Food and Drug Administration (FDA) to understand the characteristics of the novel SARS CoV-2 virus and clinical progression of COVID-19, and to provide rigorously developed evidence regarding the diagnosis, treatment, and prevention of the disease. To that end, the EA provides a unique venue for participants to crowdsource real-world evidence (RWE), instead of data, which is the focus of many case studies in this publication; investigate a prioritized list of scientific questions; and compare results without researchers having to expend capital to procure and maintain external datasets. The added benefit of this approach is that researchers can exchange knowledge and ideas corroboratively without exposing themselves to privacy breaches or dealing with consent or control issues with data—concerns raised by the research community in the progenitor publication to this one. Given the urgency to attend to the rapidly evolving COVID-19 crisis, EA used existing information-sharing infrastructure, research protocols, and the prestige of its host organizations, the Reagan-Udall Foundation and Friends of Cancer Research, to quickly attain critical mass and jumpstart its work. Work within the EA is guided by a code of conduct that emphasizes a commitment to scientific integrity, ruthless transparency, and respect for individual privacy. While the EA began with workstreams in COVID-19 therapeutics and diagnostics, it has recently expanded to COVID-19 vaccines and post-acute sequelae of SARS-CoV-2. It is also being leveraged to explore possibilities for the development of novel treatments in substance use disorder. Nonetheless, irrespective of use case, the EA’s approach to information sharing offers a model for how concerns about privacy risks and loss of competitive advantage that often stymie data sharing efforts can be addressed.

Case Study at-a-Glance: COVID-19 Evidence Accelerator.

Defining Real-World Data and Real-World Evidence.

The COVID-19 Evidence Accelerator is a multi-stakeholder collaborative created at the request of the U.S. FDA in April 2020 to understand and address the rapidly developing COVID-19 pandemic through the sharing and leveraging of RWE (FDA, 2020). The EA is managed through a partnership between the Reagan-Udall Foundation (Foundation) and Friends of Cancer Research (Friends). Established by Congress as an independent 501(c)(3), the Foundation uses its neutral position to facilitate dialogue between the FDA and other public and private entities. Comparably, Friends is a nonprofit think tank that seeks to accelerate the discovery and development of new cancer treatments through public-private convenings and research partnerships.

At the time of writing, the EA consists of 230 participating organizations engaged in any or all of its three workstreams. The Diagnostic Evidence Accelerator workstream focuses on addressing diagnostic and serological questions, such as those related to the real-world performance of COVID-19 diagnostic tests. The Therapeutic Evidence Accelerator workstream is devoted to expediting the identification of effective therapies for mitigating COVID-19 symptoms. The Vaccine Evidence Accelerator focuses on questions of vaccine performance. Representation in the collaborative spans the health care ecosystem: health care systems, national insurers, health care technology companies, pharma and biopharmaceuticals, laboratories, academics, and various branches of the federal government engage and share their expertise within the EA. Participating organizations range in size, have access to diverse sources of data, and serve diverse populations.

In addition, the EA operates within a larger national data community by interfacing with FDA-adjacent initiatives, such as the Sentinel Program, Biologics Safety and Effectiveness System, and the National Evaluation System for Health Technology; domestic activities, such as the Patient-Centered Outcomes Research Institute (PCORI); and interacts with international forums, such as the Observational Health Data Science and Informatics program (see Figure 3 ).

Real-World Data for COVID-19 Ecosystem. SOURCE: Roe, L., A. Abernethy, J. Franklin, E. Sigal, J. Allen, A. Bhat, C. Rodriguez-Watson, and S. Winckler. 2021. Accelerating Evidence Generation by Convening Diverse Stakeholders Across the Real-World Data (more...)

At its inception, the EA was not externally financed, but in July 2020, the project secured support from a federal grant and a private foundation. As activities of the EA have expanded, the EA is seeking additional funding that will support additional staff to advance new projects.

• DESCRIPTION

The EA convenes experts via Lab Meetings to rapidly share information related to real-world studies of COVID-19. During the Lab Meetings, participants share preliminary findings on research, data analytics, and methods relevant to addressing the COVID-19 pandemic in three discrete areas (see Figure 4 ). In addition, findings from parallel analyses (described below) are conveyed to this broad community. EA Lab Meetings provide a “safe collaborative space” for key players (nearly 300 organizations, at the time of writing) across the health data ecosystem to assimilate and evaluate data generated from across the country. Given the importance of information sharing in the wake of the COVID-19 pandemic, meeting summaries are posted on the EA website ( Reagan-Udall Foundation and Friends of Cancer Research, 2021 ).

COVID-19 Evidence Accelerator Work Streams. SOURCE: Reagan-Udall Foundation and Friends of Cancer Research. 2021. COVID-19 Evidence Accelerator. Available at: https://evidenceaccelerator.org (accessed November 26, 2021).

The second main EA activity is gathering various data analytics experts to explore discrete research questions across a variety of real-world data sources (e.g., insurance claims, EHRs, registries). In the parallel analysis approach, analytic partners (Accelerators) contribute to the rapid development of master protocols to illustrate the use of various treatments or diagnostics, as well as characterize the natural history of certain components of COVID-19. Some of these protocols, particularly those around several potential COVID-19 therapeutics, have been reported on EA’s website and are hosted on a cloud-based file storage site ( Reagan-Udall Foundation and Friends of Cancer Research, 2020 ). Master protocols enable parallel analyses of the same question across different data systems to quickly test reproducibility of results. Unlike most of the cases highlighted in this Special Publication, data exchange is not a feature of the EA. Hence, there is no need for a data coordinating center or data use agreements (DUAs) with research partners, although research partners who engage in data sharing external to the EA may employ DUAs. Each analytic partner applies the master protocol to their own data and performs their own analysis. Therefore, only aggregated results are shared among collaborators for knowledge generation. The benefits of this method are that it mitigates privacy risks and addresses concerns about the loss of competitive advantage that can arise from sharing identifiable data points.

To render results in an easily comparable format and reduce the burden of data handling, each master protocol consists of detailed analysis plans and uniform data tables, ensuring results are synthesized and presented uniformly. A key undertaking common to all three workstreams is to identify appropriate common data elements upfront, which contributes to the acceleration capability.

Outside of COVID-19, this approach was originally piloted to explore population characteristics, outcomes, and novel endpoints for cancer treatments across multiple data sets. The EA has recently expanded to include post-acute sequelae of SARS–CoV-2 and an oncology working group, which are exploring the feasibility to evaluate potential differences in response to cancer treatments for patients who have previously had COVID-19. The EA is also being leveraged to explore possibilities for the development of novel treatments in substance use disorder.

An outcome of this two-fold work (convening experts and pursuing discrete regulatory questions) has been the development of principles (see Figure 5 ) to guide the EA’s work, which underscore the importance of producing results that are reliable, credible, and usable by regulators and the health care community.

COVID-19 Evidence Accelerator Principles. SOURCE: Reagan-Udall Foundation and Friends of Cancer Research. 2021. COVID-19 Evidence Accelerator. Available at: https://evidenceaccelerator.org (accessed November 26, 2021).

EA’s ethos can be characterized as a “coalition of the willing.” There is no minimum time commitment or fee to participate in the collaborative, and organizations can enter and exit at will. While, on the periphery, health systems may have pre-existing DUAs with third-party analytics companies, collaborations within the EA are not governed by legal documents or a codified decision-making process. Projects presented for IRB review under the EA have been determined to be exempt. In the absence of an enforcement mechanism, EA abides by the aforementioned set of principles that recognize the sense of urgency “to act fast” without sacrificing data privacy and scientific integrity. Another key operational tenet is embracing convergence and discordance to facilitate understanding about the nuances of the underlying datasets, such as how are the data gathered, the context in which they are gathered, and methods and perspectives of interpreting the data to the extent the data is harmonized.

While Friends and the Foundation guide and provide overall support for the project’s activities, leadership for identifying research questions and resolving issues come about organically. The EA invites collaborators’ input into decisions about which questions should be of priority and how and where resources should be allocated. Final decisions about priority scientific questions rest with the FDA.

The EA is unique in part because of the mutually reinforcing relationship between the value it presents to collaborators and the factors contributing to its success. Born out of the critical need to understand the rapidly evolving natural history of COVID-19, the EA offers a frictionless knowledge sharing environment in which collaborators can share results quickly without being encumbered by bureaucracy. The benefit is derived from leveraging the tools and cachet of the EA’s host organizations (Friends, Foundation, and the FDA). Given the urgency to rapidly construct a data sharing apparatus to keep pace with the quickly evolving nature of the pandemic, the project builds upon the data-sharing and analysis efforts of the oncology-related pilot studies conducted by Friends and numerous data partners. For example, the experience of Friends offered a model in how to formulate common research protocols and apply a parallel learning model. The EA’s data-sharing infrastructure borrows from the groundwork laid by the Foundation’s post-market safety surveillance work as well. EA leaders attribute these elements as key to the project’s success.

The prefabricated infrastructure of the EA coupled with the urgency of COVID-19 endows the effort with momentum by attracting a multitude of research partners who bring to bear their capabilities and expertise. It is noteworthy to acknowledge that the credibility of the FDA, Friends, and the Foundation created an added stimulus for organizations to join. EA organizers stated how the project quickly became a favorable venue for COVID-19 researchers, especially early in the pandemic (April–May 2020), which spurred others to join for fear of regret of missing an advantageous opportunity.

In turn, the cumulative strengths of the research partners have helped the collaborative develop stable research practices and rigorously generated evidence at a time when the quality and credibility of notable scientific studies of COVID-19 were being challenged. These aspects enhance the attractiveness of the EA, setting it apart as a low-cost, low-risk information sharing solution for entities concerned with the steep financial investment to obtain and maintain high-quality data, which has been noted as a deterrent to data sharing ( Whicher et al., 2020 ).

• FUTURE DIRECTIONS

In addition to capitalizing on the resources and efforts of others, EA leaders underscore the significance of a “just do it” mentality, which at times can serve as the best antidote to the inertia of the health care industry. While early attempts to launch the project included missteps, through an iterative process driven by a willingness to learn from others and a commitment to transparency, EA leaders were able to reach a workable steady state within five months. For example, project leaders learned that a step-wise approach to analysis allowed the EA to review data and push out preliminary results to a wider audience. One of the interviewees for this case study aptly observed, “There are decisions being made every day based on limited evidence, so if we can do something, anything, to help we will do it.” While the statement was made in reference to COVID-19 decision making, the kernel of this statement is at the heart of accelerating continuous learning in health and health care and guides EA’s operations to this day as it catalogues lessons learned and tackles new research questions.

• Cite this Page National Academy of Medicine; The Learning Health System Series; Grossmann C, Chua PS, Ahmed M, et al., editors. Sharing Health Data: The Why, the Will, and the Way Forward. Washington (DC): National Academies Press (US); 2022. 5, CASE STUDY: COVID-19 EVIDENCE ACCELERATOR.
• PDF version of this title (1.5M)

In this Page

Recent activity.

• CASE STUDY: COVID-19 EVIDENCE ACCELERATOR - Sharing Health Data CASE STUDY: COVID-19 EVIDENCE ACCELERATOR - Sharing Health Data

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

Connect with NLM

National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894

Web Policies FOIA HHS Vulnerability Disclosure

Help Accessibility Careers

An official website of the United States government

Official websites use .gov A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS A lock ( Lock Locked padlock icon ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

• Publications
• Account settings
• Advanced Search
• Journal List

A case–case study on the effect of primary and booster immunization with China-produced COVID-19 vaccines on prevention of pneumonia and viral load among vaccinated persons infected by Delta and Omicron variants

Ai-bin wang, zhao-hui qian, fu-zhen wang, chang huang, hai-feng wang, yan-yang zhang, jing-jing pan, ming-xia lu, chang-shuang wang, lance everett rodewald, zun-dong yin, xuan-yi wang, yi-ming shao.

• Author information
• Article notes
• Copyright and License information

Zun-Dong Yin [email protected] National Immunization Program, Chinese Center for Disease Control and Prevention, No.27 Nanwei Road, Xicheng District, Beijing, People’s Republic of China;

Xuan-Yi Wang [email protected] Key Laboratory of Medical Molecular Virology of MoE & MoH, Shanghai Institute of Infectious Disease and Biosecurity, Shanghai, People’s Republic of China, Institutes of Biomedical Sciences, Shanghai Medical College, Fudan University, 138 Yi Xue Yuan Rd, Shanghai, People’s Republic of China;

Zhi-Yin Wu [email protected] Development Center for Medicine and Science &Technology, National Health Commission, China No.9 Chegongzhuang Street, Xicheng District, Beijing, China;

Yi-Ming Shao [email protected] National Center for AIDS/STD Control and Prevention, Chinese Center for Disease Control and Prevention Beijing, China No.155 Changbai Road, Changping District, Beijing, China

Dan Wu and Ying Ye contributed equally to this work.

Received 2022 Apr 25; Accepted 2022 Jul 14; Collection date 2022.

This is an Open Access article distributed under the terms of the Creative Commons Attribution License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Using a three-prefecture, two-variant COVID-19 outbreak in Henan province in January 2022, we evaluated the associations of primary and booster immunization with China-produced COVID-19 vaccines and COVID-19 pneumonia and SARS-CoV-2 viral load among persons infected by Delta or Omicron variant. We obtained demographic, clinical, vaccination, and multiple Ct values of infections ≥3 years of age. Vaccination status was either primary series ≥180 days prior to infection; primary series <180 days prior to infection, or booster dose recipient. We used logistic regression to determine odds ratios (OR) of Delta and Omicron COVID-19 pneumonia by vaccination status. We analysed minimum Ct values by vaccination status, age, and variant. Of 826 eligible cases, 405 were Delta and 421 were Omicron cases; 48.9% of Delta and 19.0% of Omicron cases had COVID-19 pneumonia. Compared with full primary vaccination ≥180 days before infection, the aOR of pneumonia was 0.48 among those completing primary vaccination <180 days and 0.18 among booster recipients among these Delta infections. Among Omicron infections, the corresponding aOR was 0.34 among those completing primary vaccination <180 days. There were too few (ten) Omicron cases among booster dose recipients to calculate a reliable OR. There were no differences in minimum Ct values by vaccination status among the 356 Delta cases or 70 Omicron cases. COVID-19 pneumonia was less common among Omicron cases than Delta cases. Full primary vaccination reduced pneumonia effectively for 6 months; boosting six months after primary vaccination resulted in further reduction. We recommend accelerating the pace of booster dose administration.

KEYWORDS: COVID-19 vaccine, Ct value, case-case study, real-world study, booster immunization

Introduction

As of 21 February 2022, World Health Organization (WHO) data showed that there were nearly 500 million COVID-19 cases and over 6 million deaths reported globally [ 1 ]. COVID-19 vaccines are an important means for preventing COVID-19 and its related morbidity and mortality. Real-world study of COVID-19 vaccines, at home and abroad, has been showing that while none of the available COVID-19 vaccines completely prevent COVID-19 infection, all are significantly effective for preventing hospitalization, severe illness, and death caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection [ 2–5 ]. Emergence of new variants of concern demands continuous evaluation of vaccines and their impact. WHO listed the B.1.1.529 (Omicron) variant as a Variant of Concern (VOC) on 26 November 2021, and since then, Omicron rapidly became the globally dominant variant due to its higher transmissibility and partial immune escape, although with lower virulence compared with the ancestral strain and other VOCs [ 6–8 ]. China has adopted the strict epidemic prevention and control measures of “external prevention of importation, internal prevention of rebound,” leading to low levels of local transmission, with all local transmission being associated with importation outbreaks. In 2021, most COVID-19 outbreaks were caused by the Delta variant, and in 2022, most have been Omicron outbreaks.

Important knowledge gaps are whether primary and booster immunization with China-produced COVID-19 inactivated vaccines can reduce disease severity caused by the new variants and whether they can reduce the viral load during infection. Population immunity thus far in China is only vaccine-induced due to the paucity of local transmission that would lead to hybrid immunity. One study in Jiangsu showed that primary vaccination and booster doses using China-produced inactivated COVID-19 vaccines reduced risk of Delta infection progressing from mild to severe and critical severe COVID-19 [ 9 ].

In January 2022, a local Delta-variant outbreak occurred in Zhengzhou and Xuchang Yuzhou (hereinafter referred to as “Yuzhou”) in Henan province. Coincidentally, a local Omicron outbreak occurred in Anyang city, which is also in Henan Province. The simultaneous outbreaks provided an opportunity to evaluate associations between primary and booster immunization and pneumonia among persons infected with Delta or Omicron variants, and to assess association of vaccination and viral load during infection. We reported our evaluations.

Setting and outbreak

The setting was three cities in Henan province – Zhengzhou, Yuzhou, and Anyang. As of 1 January 2022, prior to the outbreak, full-series primary vaccination coverage levels were 87.22%, 94.67%, and 80.28% of whole populations in Zhengzhou, Yuzhou, and Anyang, respectively. The vaccines used in the outbreak setting included two inactivated vaccines (91.80% of doses), BBIBP-CorV, produced by Sinopharm and CoronaVac, produced by Sinovac; one recombinant protein vaccine (7.53% of doses), ZF2001, produced by Anhui Zhifei Longcom; and one adenovirus type-5 (Ad5)-vectored vaccine (0.67% of doses), injectable Convidecia, produced by CanSinoBio. During the time of the Henan outbreak, COVID-19 antivirals were not licensed or in use.

On 2 January 2022, a ceramics factory worker in Yuzhou city was diagnosed with SARS-CoV-2 infection after a routine PCR test prior to hospital admission for surgery came back positive. As an investigation progressed, additional infections were detected in Yuzhou and Zhengzhou city (the Zheng-Yu transmission chain). On 4 January, lock down was implemented hierarchically to block transmission in the two cities. The Zheng-Yu transmission chain lasted until January 19 and ultimately consisted of 500 Delta (B.1.617.2) strain infections.

On 8 January 2022, an employee of a medical device company and a middle school student were confirmed PCR positive for SARS-CoV-2 infection when seeking health care in Anyang city hospitals (the Anyang transmission chain). Two days later, lock-down was implemented hierarchically to block transmission in the communities and the middle school. Eventually, 468 Omicron (B.1.1.529.1) infections were identified within the Anyang transmission chain, which lasted until January 29.

Study design and data sources

We used a case-case study [ 10 ] design to evaluate the association between vaccination and pneumonia among persons infected with Delta or Omicron variants. We used descriptive statistics to analyse the minimum Ct value obtained during the time between the first RT–PCR positive test and the end of hospitalization (a proxy for maximum viral load) among infected individuals by age, variant, and vaccination status, explore the association of vaccination and PCR (RT–PCR) cycle threshold (Ct) values in Delta and Omicron infections.

SARS-CoV-2 infections

We obtained data from the National Notifiable Disease Reporting System (NNDRS) on all individuals ≥3 years old diagnosed with SARS-CoV-2 infection. Data included age, gender, identity card (ID) number (for linking across data bases), and the most severe clinical classification between 2 and 23 January. The subjects in our study were close contacts of laboratory-proved infections and therefore all had known exposure to SARS-CoV-2.

Immunization histories

We obtained vaccination records from the National Immunization Information System, linked through infection ID to national ID, which indexes the immunization information system.

At least two different PCR test kits, approved by China’s National Medical Products Administration (i.e. BioGerm test kit, bioPerfectus Technologies test kit, Ediagnosis test kit, DAAN GENE test kit, XABT test kit, or Sansure Biotech test kit), were used to confirm RT–PCR positivity of each person infected. We obtained all PCR test results from local CDCs and hospitals including infection ID, sample source, test kit manufacturer, Ct value of ORF1ab and N targets, and sample collection date. To ensure that the Ct values were evaluable in our subject-level CT-value analyses, we excluded subjects for which Ct N-target values exist but Ct ORF1ab-target values were 0 or null, <10, or ≥40; subjects for which Ct ORF1ab values existed but Ct N values were 0 or null, <10, or ≥40; and subjects for which the absolute value of the difference between Ct (ORF1ab) and Ct (N) was ≥5.

Variant classification

Based on gene sequencing reports from the provincial and national laboratories, the Zheng-Yu transmission chain was caused by Delta (B.1.617.2) and the Anyang transmission chain was caused by Omicron (B.1.1.529.1).

Definitions

Vaccination status.

Based on vaccination history, individuals were classified into one of three mutually-exclusive categories: those who received full primary vaccination <180 days before diagnosis, those who received full primary vaccination ≥180 days before diagnosis, and those who had a booster dose. Full primary vaccination consisted of completion of two doses of an inactivated vaccine or one dose of the Ad5-vectored vaccine or three doses of the CHO protein recombinant vaccine 14 days or more before exposure to SARS-CoV-2, or receipt of one booster dose but within seven days of exposure. Booster dose receipt consisted of those who completed 3 doses of inactivated vaccine or 2 doses of Ad5 vaccine at least seven days prior to exposure. There was a small number (69) of individuals with the unusual vaccination status of unvaccinated or partially vaccinated; these individuals were not included in the analyses.

Severity and outcomes

Asymptomatic cases were individuals who never developed any symptoms of any type during quarantine. Mild cases had one or more subjectively mild symptoms of infection, but did not have any signs of pneumonia on imaging. Moderate cases had manifestations of pneumonia along with any symptoms of COVID-19 such as fever, cough, or other respiratory symptoms. Diagnosis of pneumonia was based on clinical symptoms and CT imaging. Chest CT imaging findings suggestive of COVID-19 pneumonia were multiple bilateral ground glass opacities, often rounded in morphology, with peripheral and lower lung distribution [ 11 ].

Severe cases met any of the following criteria: (a) respiratory distress or respiratory rate 30 breaths/minute or more, (b) oxygen saturation less than 93% at rest, (c) a ratio of arterial partial pressure of oxygen (PaO2) to fractional inspired oxygen concentration (FiO2) of <300 mmHg (with adjustment for altitudes above 1000 metres), or (d) patients with pneumonia having >50% lesion progression within 24–48 h seen in pulmonary imaging. Critical Cases met any of the following criteria: (a) respiratory failure or need for mechanical ventilation, (b) shock, or (c) other organ failure that required monitoring and treatment in the ICU.

Statistical analysis

For multivariate analysis of the associations between the three vaccination statuses (full primary vaccination ≥180 days and no booster dose, full primary vaccination <180 days, booster dose) and the three main outcomes (symptomatic COVID-19, pneumonia, and severe/critical COVID-19), outcomes were dependent variables, vaccination statuses were independent variables, and gender, age group (3–17 years old, 18–49 years old, and ≥50 years old), and presence of underlying disease(s) were considered potentially confounding variables. The age groupings were selected to reflect differences in vaccine recommendations and distribution of comorbidities: booster doses were not recommended for the youngest group; 95% of comorbidities were concentrated in the oldest group. To ensure common-direction assessments of vaccine effectiveness, we used the theoretically least protective vaccination status – full primary vaccination ≥180 days and no booster dose – as the reference group. We used conditional logistic regression-determined odds ratios (ORs) and adjusted ORs (aORs) of pneumonia to compare the reference group to the full primary vaccination <180 days group and the booster group, stratified by gender, age group, and presence of underlying conditions. Reductions in outcome risk were calculated by 1-aOR. For the minimum Ct value analysis, we obtained all Ct values for infected individuals and determined the minimum Ct value for each person of their multiple consecutive nucleic acid tests. We compared minimum Ct values in the vaccination groups stratified by age group after matching individuals by PCR reagent manufacturer. We used the Kruskal Wallis rank sum test to assess statistical significance of differences in minimum Ct value by vaccination group. SAS (version 9.4; SAS Institute) was used for matching and statistical analyses.

Ethical review

COVID-19 is a Level-two infectious disease being managed as a Level-one (highest) infectious disease. Investigation of outbreaks is a responsibility of the institutions with which the authors are affiliated. China CDC and the CDC system have access to individual-level disease and vaccination data in their routine jobs. Analytic data sets were de-identified. Ethical Review Committee approval for this routine public health work is not required.

Role of the funding source

Investigators and authors were employed by the funder, and therefore, the funder participated in all aspects of the study, including conceptualization, design, data collection, analysis, interpretation, drafting the manuscript, and the decision to submit the manuscript for publication.

During the study period in the study setting, there were 946 individuals aged 3 years or older who were diagnosed by PCR as having SARS-CoV-2 infection. There were 69 infected individuals who were unvaccinated or partially vaccinated, and among them, 21 (30.43%) had underlying conditions. There were 826 infected individuals with CT imaging results who had completed full primary vaccination or received a booster dose and who were, therefore, eligible for the case-case study: 405 were infected by Delta and 421 were infected by Omicron ( Figure 1 ). The male: female ratios for Delta and Omicron infections were 1:1.37 and 1:1.31, respectively, and 48.89% and 19.00% of Delta and Omicron infections, respectively, met COVID-19 pneumonia criteria ( Table 1 ). All individuals under 18 years old had received full primary vaccination <180 days prior to infection. Table 2 shows the distribution of pneumonia by age group and vaccination status.

Case-case study flow chart and study subjects.

Demographic characteristics, vaccination status, and clinical classification of the case-case study subjects, Henan, 2 January 2022–23 January 2022.

COVID-19 pneumonia, stratified by age group, vaccination status and VOC, Henan, 2 January 2022–23 January 2022.

Pneumonia by variant and vaccination status

The percent of Delta variant infections that were pneumonia ranged by vaccination group from 30.77% to 69.17%, and the percent Omicron infections that were pneumonia ranged from 14.76% to 36.71%. Among people ≥50 years old who had full primary vaccination ≥180 days before Delta infection, 80.77% had pneumonia. Among people ≥50 years old who had full primary vaccination ≥180 days before Omicron infection, 42.31% had pneumonia ( Table 2 ).

Factors associated with pneumonia

Table 3 shows associations between gender, age group, comorbidities, and vaccination status and pneumonia for Delta and Omicron cases. Univariate and multivariate analyses showed that variant, age group, and vaccination status were statistically significantly associated with pneumonia. Multivariate analysis showed that compared with Delta, the odds ratio (OR) of association with pneumonia in the Omicron group was 0.34 (95%CI: 0.24–0.48).

Factors associated with Delta and Omicron variant pneumonia.

Note: *Indicates that the number of cases is too small to make a valid estimate.

With analyses restricted to Delta infections, univariate analysis showed that age group, presence of one or more comorbidities, and vaccination status were statistically significantly associated with pneumonia. For Delta infections by age group, compared with <18 years old, the ORs of having COVID-19 pneumonia among 18–49 years and ≥50 years were 11.52 (95%CI: 4.00–33.15) and 29.27 (95%CI: 10.10–84.80), respectively, and compared with full primary vaccination ≥180 days, the ORs of having pneumonia among full primary vaccination <180 days and booster dose groups were 0.34 (95%CI: 0.21–0.54) and 0.20 (95%CI: 0.10–0.38), respectively ( Table 3 ).

Multivariate analysis among Delta variant infections showed that age group and vaccination status were statistically significantly associated with pneumonia. Compared with <18-year-olds, the OR of having pneumonia among the ≥50 years age group was 25.70 (95%CI: 8.56–77.10). Compared with full primary vaccination ≥180 days, the ORs of having pneumonia among primary vaccination ≤180 days and booster dose groups were 0.48 (95%CI: 0.21–0.64) and 0.18 (95%CI: 0.09–0.35) ( Table 3 ).

With analyses restricted to Omicron infections, univariate analysis showed that age group, presence of comorbidities, and vaccination status were statistically significantly associated with pneumonia. Compared with <18-year-olds, the OR of having pneumonia among ≥50-year-olds was 3.99 (95%CI: 2.02–7.90). Compared with no comorbidities, the OR of having pneumonia among those with one or more comorbidities was 4.40 (95%CI: 2.09–9.25). Compared with full primary vaccination ≥180 days, the OR of having pneumonia among those with primary vaccination ≤ 180 days was 0.30 (95%CI: 0.17–0.52) ( Table 3 ).

Multivariate analysis among Omicron infections showed that comorbidities and vaccination status were statistically significantly associated with pneumonia. Compared with no comorbidities, the OR of having pneumonia among those with one or more comorbidities was 2.95 (95%CI: 1.16–7.63). Compared with full primary vaccination ≥180 days, the OR of having pneumonia among those with primary vaccination ≤ 180 days was 0.34 (95%CI: 0.17–0.67) ( Table 3 ).

Viral load (Ct value analysis)

There were 2140 samples that met all inclusion and no exclusion criteria and were included in the Ct value analysis. Among the 946 infected individuals, 426 used the same PCR manufacturer reagent, 356 were Delta variant infections and 70 were Omicron infections ( Figure 2 ).

Minimum Ct value study subjects.

Among those infected with the Delta variant, the medians of the minimum Ct values (ORF1ab target) among the primary vaccination ≥180 days group, the primary vaccination <180 days, and the booster dose group were 21.7, 22.0, and 23.7, respectively. The medians of the minimum Ct values (N target) were 20.9, 21.0, and 22.0, respectively. There were no statistically significant differences.

Among Omicron variant infections, the medians of the minimum Ct value (ORF1ab target) in the primary vaccination ≥ 180 days group and the primary vaccination <180 days group were both 23.0; the minimum Ct values (N target) were consistent with Ct values (ORF1ab target); there was no significant difference between groups. Since there was only one data-eligible infection in the booster vaccination group, it was unable to be matched by the reagent manufacturer and therefore not included in the Ct value analysis. There were no significant differences in minimum Ct values by age group and variant ( Table 4 ).

Minimum Ct values of Delta and Omicron variant infections stratified by age group and vaccination status.

Our case-case study showed Omicron variant infection was 66% less likely to result in pneumonia than Delta variant infection. For Delta variant infections, compared with full primary vaccination at least six months before infection, full primary vaccination within 6 months of infection reduced the risk of getting pneumonia by 52%, and receipt of a booster dose reduced the risk of pneumonia by 82%. For Omicron infections, full primary vaccination within 6 months reduced the risk of getting pneumonia by 66%. There were too few Omicron cases in the booster group to calculate a valid odds ratio of pneumonia. We found no statistically significant association between viral load and vaccination status, although there was a trend toward lower viral loads with an increase in vaccination status.

There were no severe or critically-severe Omicron infections in this province-wide outbreak, and risk of pneumonia in all age groups and all vaccination statuses was lower among Omicron infections than Delta infections. These findings are consistent with reduced severity of Omicron cases in a study in a hospital in South Africa [ 12 ], in an England cohort study [ 13 ], and in a two-year observational study in southern Sweden [ 14 ].

The first six months following full primary series vaccination was associated with reduced risk of getting pneumonia and booster dose after six months of primary vaccination was associated with reduced risk of pneumonia in our study. This finding is consistent with an observational study of BNT162b2 mRNA COVID-19 vaccine [ 15 ] that showed that a third dose is effective in protecting individuals against severe COVID-19-related outcomes compared with receiving two doses at least 5 months before exposure to SARS-CoV-2.

That we found a non-statistically significant trend in the age-group analyses in minimum Ct value by vaccination status may reflect the small sample size in our study. The trend we found is consistent with a study showing that the decreased viral load after receipt of Pfizer’s mRNA vaccine began to diminish two months after primary series vaccination and did not persist after six months unless a booster dose was administered [ 18 ]. Overseas studies have shown that maximum viral loads in Omicron-infected patients are lower than in Delta-infected patients, but we did not find differences by variant. It is possible that natural infection after vaccination leads to higher neutralization antibody titres (hybrid immunity) than primary and booster immunity [ 19 , 20 ] and leads to viral load differences by variant. Since there were relatively few natural infections in China during or prior to our study, a lack of hybrid immunity could have led to the lack of difference in the maximum viral load between Omicron-infected patients and Delta-infected patients in our study.

In the epidemic spread in Henan, only 15% of the Delta-infected individuals were <18 years old. In contrast, 45% of the Omicron infected individuals were <18 years old. China vaccinated children <18 years old later than adults (starting mid-July 2021 versus December 2020), which may have confounded the relation between vaccination age and variant. Similarly, 16.1% of the Delta-infected individuals in our study received booster doses, while only 2.4% of the Omicron-infected people received booster doses. Since children were vaccinated later and during this study period were not recommended for booster doses, the difference is most likely related to timing of vaccination policy.

Our study has several strengths. The Delta and Omicron outbreaks were simultaneous and in a single province of China, allowing differences in Delta and Omicron infections to be observed in similarly-vaccinated populations. The nearly-complete lack of previous local transmission in Henan means that immunity in our study was entirely vaccine-induced and not hybrid immunity. This allowed comparison of vaccine effectiveness against pneumonia and viral load by the two variants in a level playing field of pure vaccine-induced immunity.

Our study has limitations. First, our study was based on cases only (a case-case study), and so the results are not vaccine effectiveness against pneumonia, but an estimation of the association between vaccination status and pneumonia. The association is a measure of impact on top of an unmeasured baseline vaccine effectiveness. The design we used has been used in vaccine effect evaluations for influenza vaccines [ 10 , 16 ] and COVID-19 vaccines [ 17 ]. Second, because the numbers of people infected with Delta and Omicron strains were small, the 95% confidence intervals were wide and we were unable to conduct desirable subgroup analyses such as relative VE by specific comorbidity. The comorbidities in our subjects were high blood pressure, diabetes, cerebral vascular disease, coronary heart disease, asthma, emphysema, chronic bronchitis, lung cancer, chronic liver disease, liver cancer, chronic kidney disease, immunodeficiency, AIDS, tuberculosis, and pregnancy. Third, there were only 10 Omicron cases that received a booster dose. Therefore, we could not evaluate ORs of booster dose associated with pneumonia among Omicron infections. Fourth, collection of Ct values was not part of a predetermined study design, and so the number of nucleic acid tests, reagent manufacturers, and testers varied. Although our study used one of the most commonly used reagents, Ct values can also be affected by the extraction technology and the test personnel [ 21 ]. The number of samples included in Omicron analysis was small (70 in total), thus affecting the stability of the minimum Ct value by vaccination status in Omicron patients.

In conclusion, our analysis of the association between vaccination status and pneumonia among Delta and Omicron infections in a Henan province outbreak found that Omicron infection is less likely to lead to pneumonia than Delta infection. The first six months after primary vaccination with China-produced vaccines and administration of booster doses after six months were associated with much lower risk of pneumonia. We found no difference in Ct values by vaccination status or variant. We recommend continuing to increase primary series coverage and accelerating use of timely booster doses, six months after primary vaccination. Monitoring vaccine use, safety, and effectiveness by vaccine type and schedule is essential for keeping COVID-19 vaccination policy up-to-date and as effective as possible.

Acknowledgements

We are very grateful for the data collection efforts by local CDC and infectious hospital staff of Zhengzhou, Yuzhou, and Anyang cities in Henan province.

Funding Statement

This work was supported by the Operation of Public Health Emergency Response Mechanism of Chinese Center for Disease Control and Prevention (131031001000200001).

Author contributors

ZDY, ZYW, XYW, XYL and YMS acquired funding and contributed to the study’s conception and design. ABW, RZ, ZHQ, FZW, HZ, ZJ and YY conceptualized the study and prepared the original study protocol, which was subsequently reviewed by LER. DW, LT, and CH developed the statistical methods. DW, CH and XYW wrote and tested the SAS code for the data analysis and drafted the manuscript. YY, HFW, YYZ, JJP, YFL, MXL, CSW and YTM collected data from local CDCs and abstracted the data. DW, LT, LER ZJA, ZDY and YMS interpreted the results and revised the manuscript and critically reviewed the manuscript. All authors approved the final submitted version.

• 1. World Health Organization . Available from: https://covid19.who.int/ .
• 2. Wu D, Zhang Y, Tang L, et al. . Effectiveness of inactivated COVID-19 vaccines against symptomatic, pneumonia, and severe disease caused by the Delta variant: real world study and evidence - China, 2021. China CDC Wkly. 2022;4(4):57–65. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 3. Jara A, Undurraga EA, Gonzalez C, et al. . Effectiveness of an inactivated SARS-CoV-2 vaccine in Chile. N Engl J Med. 2021;385(10):875–884. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 4. Kang M, Yan Li YY, Sun L, et al. . Effectiveness of inactivated COVID-19 vaccines against COVID-19 pneumonia and severe illness caused by the B.1.617.2 (Delta) variant: evidence from an outbreak in Guangdong, China: A Cohort Study. Ann Intern Med. 2021;175(4):533–540. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 5. Embi PJ, Levy ME, Naleway AL, et al. . Effectiveness of 2-dose vaccination with mRNA COVID-19 vaccines against COVID-19-associated hospitalizations among immunocompromised adults - nine states, January-September 2021. MMWR Morb Mortal Wkly Rep. 2021;70(44):1553–1559. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 6. Sheikh A, McMenamin J, Taylor B, et al. . SARS-CoV-2 Delta VOC in Scotland: demographics, risk of hospital admission, and vaccine effectiveness. Lancet. 2021;397(10293):2461–2462. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 7. Wall EC, Wu M, Harvey R, et al. . Neutralising antibody activity against SARS-CoV-2 VOCs B.1.617.2 and B.1.351 by BNT162b2 vaccination. Lancet. 2021;397(10292):2331–2333. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 8. Wolter N, Jassat W, Walaza S, et al. . Early assessment of the clinical severity of the SARS-CoV-2 Omicron variant in South Africa: a data linkage study. Lancet. 2022;399(10323):437–446. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 9. Hu Z, Tao B, Li Z, et al. . Effectiveness of inactivated COVID-19 vaccines against severe illness in B.1.617.2 (Delta) variant-infected patients in Jiangsu, China. Int J Infect Dis. 2022;116:204–209. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 10. Godoy P, Romero A, Soldevila N, et al. . Influenza vaccine effectiveness in reducing severe outcomes over six influenza seasons, a case-case analysis, Spain, 2010/11 to 2015/16. Euro Surveill. 2018;23(43). [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 11. Organizaiton , W.H. WHO COVID-19: Case Definitions. 2020.
• 12. Abdullah F, Myers J, Basu D, et al. . Decreased severity of disease during the first global Omicron variant COVID-19 outbreak in a large hospital in Tshwane, South Africa. Int J Infect Dis. 2021;116:38–42. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 13. Krutikov M, Stirrup O, Nacer-Laidi H, et al. . COVID-19 Genomics UK consortium. Outcomes of SARS-CoV-2 omicron infection in residents of long-term care facilities in England (VIVALDI): a prospective, cohort study. Lancet Healthy Longev. 2022;3(5):e347–e355. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 14. Kahn F, Bonander C, Moghaddassi M, et al. . Risk of severe COVID-19 from the Delta and Omicron variants in relation to vaccination status, sex, age and comorbidities - surveillance results from southern Sweden, July 2021 to January 2022. Euro Surveill. 2022;27(9). [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 15. Barda N, Dagan N, Cohen C, et al. . Effectiveness of a third dose of the BNT162b2 mRNA COVID-19 vaccine for preventing severe outcomes in Israel: an observational study. Lancet. 2021;398(10316):2093–2100. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 16. Puig-Barbera J, Diez-Domingo J, Arnedo-Pena A, et al. . Effectiveness of the 2010-2011 seasonal influenza vaccine in preventing confirmed influenza hospitalizations in adults: a case-case comparison, case-control study. Vaccine. 2012;30(39):5714–5720. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 17. Kislaya I, Rodrigues EF, Borges V, et al. . Comparative effectiveness of coronavirus vaccine in preventing breakthrough infections among vaccinated persons infected with Delta and Alpha variants. Emerg Infect Dis. 2022;28(2):331–337. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 18. Levine-Tiefenbrun M, Yelin I, Alapi H, et al. . Viral loads of Delta-variant SARS-CoV-2 breakthrough infections after vaccination and booster with BNT162b2. Nat Med. 2021;27(12):2108–2110. [ DOI ] [ PubMed ] [ Google Scholar ]
• 19. Pilz S, Theiler-Schwetz V, Trummer C, et al. . SARS-CoV-2 reinfections: overview of efficacy and duration of natural and hybrid immunity. Environ Res. 2022;209:112911. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 20. Sigal A. Milder disease with Omicron: is it the virus or the pre-existing immunity? Nat Rev Immunol. 2022;22(2):69–71. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 21. England, P.H . Understanding cycle threshold (Ct) in SARS-CoV-2 RT-PCR. 2020.
• View on publisher site
• PDF (1.5 MB)
• Collections

Similar articles

Cited by other articles, links to ncbi databases.

• Download .nbib .nbib
• Format: AMA APA MLA NLM

Add to Collections

• Research article
• Open access
• Published: 04 June 2021

Coronavirus disease (COVID-19) pandemic: an overview of systematic reviews

• Israel Júnior Borges do Nascimento 1 , 2 ,
• Dónal P. O’Mathúna 3 , 4 ,
• Thilo Caspar von Groote 5 ,
• Hebatullah Mohamed Abdulazeem 6 ,
• Ishanka Weerasekara 7 , 8 ,
• Ana Marusic 9 ,
• Livia Puljak   ORCID: orcid.org/0000-0002-8467-6061 10 ,
• Vinicius Tassoni Civile 11 ,
• Irena Zakarija-Grkovic 9 ,
• Tina Poklepovic Pericic 9 ,
• Alvaro Nagib Atallah 11 ,
• Santino Filoso 12 ,
• Nicola Luigi Bragazzi 13 &
• Milena Soriano Marcolino 1

On behalf of the International Network of Coronavirus Disease 2019 (InterNetCOVID-19)

BMC Infectious Diseases volume  21 , Article number:  525 ( 2021 ) Cite this article

19k Accesses

38 Citations

14 Altmetric

Metrics details

Navigating the rapidly growing body of scientific literature on the SARS-CoV-2 pandemic is challenging, and ongoing critical appraisal of this output is essential. We aimed to summarize and critically appraise systematic reviews of coronavirus disease (COVID-19) in humans that were available at the beginning of the pandemic.

Nine databases (Medline, EMBASE, Cochrane Library, CINAHL, Web of Sciences, PDQ-Evidence, WHO’s Global Research, LILACS, and Epistemonikos) were searched from December 1, 2019, to March 24, 2020. Systematic reviews analyzing primary studies of COVID-19 were included. Two authors independently undertook screening, selection, extraction (data on clinical symptoms, prevalence, pharmacological and non-pharmacological interventions, diagnostic test assessment, laboratory, and radiological findings), and quality assessment (AMSTAR 2). A meta-analysis was performed of the prevalence of clinical outcomes.

Eighteen systematic reviews were included; one was empty (did not identify any relevant study). Using AMSTAR 2, confidence in the results of all 18 reviews was rated as “critically low”. Identified symptoms of COVID-19 were (range values of point estimates): fever (82–95%), cough with or without sputum (58–72%), dyspnea (26–59%), myalgia or muscle fatigue (29–51%), sore throat (10–13%), headache (8–12%) and gastrointestinal complaints (5–9%). Severe symptoms were more common in men. Elevated C-reactive protein and lactate dehydrogenase, and slightly elevated aspartate and alanine aminotransferase, were commonly described. Thrombocytopenia and elevated levels of procalcitonin and cardiac troponin I were associated with severe disease. A frequent finding on chest imaging was uni- or bilateral multilobar ground-glass opacity. A single review investigated the impact of medication (chloroquine) but found no verifiable clinical data. All-cause mortality ranged from 0.3 to 13.9%.

Conclusions

In this overview of systematic reviews, we analyzed evidence from the first 18 systematic reviews that were published after the emergence of COVID-19. However, confidence in the results of all reviews was “critically low”. Thus, systematic reviews that were published early on in the pandemic were of questionable usefulness. Even during public health emergencies, studies and systematic reviews should adhere to established methodological standards.

Peer Review reports

The spread of the “Severe Acute Respiratory Coronavirus 2” (SARS-CoV-2), the causal agent of COVID-19, was characterized as a pandemic by the World Health Organization (WHO) in March 2020 and has triggered an international public health emergency [ 1 ]. The numbers of confirmed cases and deaths due to COVID-19 are rapidly escalating, counting in millions [ 2 ], causing massive economic strain, and escalating healthcare and public health expenses [ 3 , 4 ].

The research community has responded by publishing an impressive number of scientific reports related to COVID-19. The world was alerted to the new disease at the beginning of 2020 [ 1 ], and by mid-March 2020, more than 2000 articles had been published on COVID-19 in scholarly journals, with 25% of them containing original data [ 5 ]. The living map of COVID-19 evidence, curated by the Evidence for Policy and Practice Information and Co-ordinating Centre (EPPI-Centre), contained more than 40,000 records by February 2021 [ 6 ]. More than 100,000 records on PubMed were labeled as “SARS-CoV-2 literature, sequence, and clinical content” by February 2021 [ 7 ].

Due to publication speed, the research community has voiced concerns regarding the quality and reproducibility of evidence produced during the COVID-19 pandemic, warning of the potential damaging approach of “publish first, retract later” [ 8 ]. It appears that these concerns are not unfounded, as it has been reported that COVID-19 articles were overrepresented in the pool of retracted articles in 2020 [ 9 ]. These concerns about inadequate evidence are of major importance because they can lead to poor clinical practice and inappropriate policies [ 10 ].

Systematic reviews are a cornerstone of today’s evidence-informed decision-making. By synthesizing all relevant evidence regarding a particular topic, systematic reviews reflect the current scientific knowledge. Systematic reviews are considered to be at the highest level in the hierarchy of evidence and should be used to make informed decisions. However, with high numbers of systematic reviews of different scope and methodological quality being published, overviews of multiple systematic reviews that assess their methodological quality are essential [ 11 , 12 , 13 ]. An overview of systematic reviews helps identify and organize the literature and highlights areas of priority in decision-making.

In this overview of systematic reviews, we aimed to summarize and critically appraise systematic reviews of coronavirus disease (COVID-19) in humans that were available at the beginning of the pandemic.

Methodology

Research question.

This overview’s primary objective was to summarize and critically appraise systematic reviews that assessed any type of primary clinical data from patients infected with SARS-CoV-2. Our research question was purposefully broad because we wanted to analyze as many systematic reviews as possible that were available early following the COVID-19 outbreak.

Study design

We conducted an overview of systematic reviews. The idea for this overview originated in a protocol for a systematic review submitted to PROSPERO (CRD42020170623), which indicated a plan to conduct an overview.

Overviews of systematic reviews use explicit and systematic methods for searching and identifying multiple systematic reviews addressing related research questions in the same field to extract and analyze evidence across important outcomes. Overviews of systematic reviews are in principle similar to systematic reviews of interventions, but the unit of analysis is a systematic review [ 14 , 15 , 16 ].

We used the overview methodology instead of other evidence synthesis methods to allow us to collate and appraise multiple systematic reviews on this topic, and to extract and analyze their results across relevant topics [ 17 ]. The overview and meta-analysis of systematic reviews allowed us to investigate the methodological quality of included studies, summarize results, and identify specific areas of available or limited evidence, thereby strengthening the current understanding of this novel disease and guiding future research [ 13 ].

A reporting guideline for overviews of reviews is currently under development, i.e., Preferred Reporting Items for Overviews of Reviews (PRIOR) [ 18 ]. As the PRIOR checklist is still not published, this study was reported following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2009 statement [ 19 ]. The methodology used in this review was adapted from the Cochrane Handbook for Systematic Reviews of Interventions and also followed established methodological considerations for analyzing existing systematic reviews [ 14 ].

Approval of a research ethics committee was not necessary as the study analyzed only publicly available articles.

Eligibility criteria

Systematic reviews were included if they analyzed primary data from patients infected with SARS-CoV-2 as confirmed by RT-PCR or another pre-specified diagnostic technique. Eligible reviews covered all topics related to COVID-19 including, but not limited to, those that reported clinical symptoms, diagnostic methods, therapeutic interventions, laboratory findings, or radiological results. Both full manuscripts and abbreviated versions, such as letters, were eligible.

No restrictions were imposed on the design of the primary studies included within the systematic reviews, the last search date, whether the review included meta-analyses or language. Reviews related to SARS-CoV-2 and other coronaviruses were eligible, but from those reviews, we analyzed only data related to SARS-CoV-2.

No consensus definition exists for a systematic review [ 20 ], and debates continue about the defining characteristics of a systematic review [ 21 ]. Cochrane’s guidance for overviews of reviews recommends setting pre-established criteria for making decisions around inclusion [ 14 ]. That is supported by a recent scoping review about guidance for overviews of systematic reviews [ 22 ].

Thus, for this study, we defined a systematic review as a research report which searched for primary research studies on a specific topic using an explicit search strategy, had a detailed description of the methods with explicit inclusion criteria provided, and provided a summary of the included studies either in narrative or quantitative format (such as a meta-analysis). Cochrane and non-Cochrane systematic reviews were considered eligible for inclusion, with or without meta-analysis, and regardless of the study design, language restriction and methodology of the included primary studies. To be eligible for inclusion, reviews had to be clearly analyzing data related to SARS-CoV-2 (associated or not with other viruses). We excluded narrative reviews without those characteristics as these are less likely to be replicable and are more prone to bias.

Scoping reviews and rapid reviews were eligible for inclusion in this overview if they met our pre-defined inclusion criteria noted above. We included reviews that addressed SARS-CoV-2 and other coronaviruses if they reported separate data regarding SARS-CoV-2.

Information sources

Nine databases were searched for eligible records published between December 1, 2019, and March 24, 2020: Cochrane Database of Systematic Reviews via Cochrane Library, PubMed, EMBASE, CINAHL (Cumulative Index to Nursing and Allied Health Literature), Web of Sciences, LILACS (Latin American and Caribbean Health Sciences Literature), PDQ-Evidence, WHO’s Global Research on Coronavirus Disease (COVID-19), and Epistemonikos.

The comprehensive search strategy for each database is provided in Additional file 1 and was designed and conducted in collaboration with an information specialist. All retrieved records were primarily processed in EndNote, where duplicates were removed, and records were then imported into the Covidence platform [ 23 ]. In addition to database searches, we screened reference lists of reviews included after screening records retrieved via databases.

Study selection

All searches, screening of titles and abstracts, and record selection, were performed independently by two investigators using the Covidence platform [ 23 ]. Articles deemed potentially eligible were retrieved for full-text screening carried out independently by two investigators. Discrepancies at all stages were resolved by consensus. During the screening, records published in languages other than English were translated by a native/fluent speaker.

Data collection process

We custom designed a data extraction table for this study, which was piloted by two authors independently. Data extraction was performed independently by two authors. Conflicts were resolved by consensus or by consulting a third researcher.

We extracted the following data: article identification data (authors’ name and journal of publication), search period, number of databases searched, population or settings considered, main results and outcomes observed, and number of participants. From Web of Science (Clarivate Analytics, Philadelphia, PA, USA), we extracted journal rank (quartile) and Journal Impact Factor (JIF).

We categorized the following as primary outcomes: all-cause mortality, need for and length of mechanical ventilation, length of hospitalization (in days), admission to intensive care unit (yes/no), and length of stay in the intensive care unit.

The following outcomes were categorized as exploratory: diagnostic methods used for detection of the virus, male to female ratio, clinical symptoms, pharmacological and non-pharmacological interventions, laboratory findings (full blood count, liver enzymes, C-reactive protein, d-dimer, albumin, lipid profile, serum electrolytes, blood vitamin levels, glucose levels, and any other important biomarkers), and radiological findings (using radiography, computed tomography, magnetic resonance imaging or ultrasound).

We also collected data on reporting guidelines and requirements for the publication of systematic reviews and meta-analyses from journal websites where included reviews were published.

Quality assessment in individual reviews

Two researchers independently assessed the reviews’ quality using the “A MeaSurement Tool to Assess Systematic Reviews 2 (AMSTAR 2)”. We acknowledge that the AMSTAR 2 was created as “a critical appraisal tool for systematic reviews that include randomized or non-randomized studies of healthcare interventions, or both” [ 24 ]. However, since AMSTAR 2 was designed for systematic reviews of intervention trials, and we included additional types of systematic reviews, we adjusted some AMSTAR 2 ratings and reported these in Additional file 2 .

Adherence to each item was rated as follows: yes, partial yes, no, or not applicable (such as when a meta-analysis was not conducted). The overall confidence in the results of the review is rated as “critically low”, “low”, “moderate” or “high”, according to the AMSTAR 2 guidance based on seven critical domains, which are items 2, 4, 7, 9, 11, 13, 15 as defined by AMSTAR 2 authors [ 24 ]. We reported our adherence ratings for transparency of our decision with accompanying explanations, for each item, in each included review.

One of the included systematic reviews was conducted by some members of this author team [ 25 ]. This review was initially assessed independently by two authors who were not co-authors of that review to prevent the risk of bias in assessing this study.

Synthesis of results

For data synthesis, we prepared a table summarizing each systematic review. Graphs illustrating the mortality rate and clinical symptoms were created. We then prepared a narrative summary of the methods, findings, study strengths, and limitations.

For analysis of the prevalence of clinical outcomes, we extracted data on the number of events and the total number of patients to perform proportional meta-analysis using RStudio© software, with the “meta” package (version 4.9–6), using the “metaprop” function for reviews that did not perform a meta-analysis, excluding case studies because of the absence of variance. For reviews that did not perform a meta-analysis, we presented pooled results of proportions with their respective confidence intervals (95%) by the inverse variance method with a random-effects model, using the DerSimonian-Laird estimator for τ 2 . We adjusted data using Freeman-Tukey double arcosen transformation. Confidence intervals were calculated using the Clopper-Pearson method for individual studies. We created forest plots using the RStudio© software, with the “metafor” package (version 2.1–0) and “forest” function.

Managing overlapping systematic reviews

Some of the included systematic reviews that address the same or similar research questions may include the same primary studies in overviews. Including such overlapping reviews may introduce bias when outcome data from the same primary study are included in the analyses of an overview multiple times. Thus, in summaries of evidence, multiple-counting of the same outcome data will give data from some primary studies too much influence [ 14 ]. In this overview, we did not exclude overlapping systematic reviews because, according to Cochrane’s guidance, it may be appropriate to include all relevant reviews’ results if the purpose of the overview is to present and describe the current body of evidence on a topic [ 14 ]. To avoid any bias in summary estimates associated with overlapping reviews, we generated forest plots showing data from individual systematic reviews, but the results were not pooled because some primary studies were included in multiple reviews.

Our search retrieved 1063 publications, of which 175 were duplicates. Most publications were excluded after the title and abstract analysis ( n = 860). Among the 28 studies selected for full-text screening, 10 were excluded for the reasons described in Additional file 3 , and 18 were included in the final analysis (Fig. 1 ) [ 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 ]. Reference list screening did not retrieve any additional systematic reviews.

PRISMA flow diagram

Characteristics of included reviews

Summary features of 18 systematic reviews are presented in Table 1 . They were published in 14 different journals. Only four of these journals had specific requirements for systematic reviews (with or without meta-analysis): European Journal of Internal Medicine, Journal of Clinical Medicine, Ultrasound in Obstetrics and Gynecology, and Clinical Research in Cardiology . Two journals reported that they published only invited reviews ( Journal of Medical Virology and Clinica Chimica Acta ). Three systematic reviews in our study were published as letters; one was labeled as a scoping review and another as a rapid review (Table 2 ).

All reviews were published in English, in first quartile (Q1) journals, with JIF ranging from 1.692 to 6.062. One review was empty, meaning that its search did not identify any relevant studies; i.e., no primary studies were included [ 36 ]. The remaining 17 reviews included 269 unique studies; the majority ( N = 211; 78%) were included in only a single review included in our study (range: 1 to 12). Primary studies included in the reviews were published between December 2019 and March 18, 2020, and comprised case reports, case series, cohorts, and other observational studies. We found only one review that included randomized clinical trials [ 38 ]. In the included reviews, systematic literature searches were performed from 2019 (entire year) up to March 9, 2020. Ten systematic reviews included meta-analyses. The list of primary studies found in the included systematic reviews is shown in Additional file 4 , as well as the number of reviews in which each primary study was included.

Population and study designs

Most of the reviews analyzed data from patients with COVID-19 who developed pneumonia, acute respiratory distress syndrome (ARDS), or any other correlated complication. One review aimed to evaluate the effectiveness of using surgical masks on preventing transmission of the virus [ 36 ], one review was focused on pediatric patients [ 34 ], and one review investigated COVID-19 in pregnant women [ 37 ]. Most reviews assessed clinical symptoms, laboratory findings, or radiological results.

Systematic review findings

The summary of findings from individual reviews is shown in Table 2 . Overall, all-cause mortality ranged from 0.3 to 13.9% (Fig. 2 ).

A meta-analysis of the prevalence of mortality

Clinical symptoms

Seven reviews described the main clinical manifestations of COVID-19 [ 26 , 28 , 29 , 34 , 35 , 39 , 41 ]. Three of them provided only a narrative discussion of symptoms [ 26 , 34 , 35 ]. In the reviews that performed a statistical analysis of the incidence of different clinical symptoms, symptoms in patients with COVID-19 were (range values of point estimates): fever (82–95%), cough with or without sputum (58–72%), dyspnea (26–59%), myalgia or muscle fatigue (29–51%), sore throat (10–13%), headache (8–12%), gastrointestinal disorders, such as diarrhea, nausea or vomiting (5.0–9.0%), and others (including, in one study only: dizziness 12.1%) (Figs. 3 , 4 , 5 , 6 , 7 , 8 and 9 ). Three reviews assessed cough with and without sputum together; only one review assessed sputum production itself (28.5%).

A meta-analysis of the prevalence of fever

A meta-analysis of the prevalence of cough

A meta-analysis of the prevalence of dyspnea

A meta-analysis of the prevalence of fatigue or myalgia

A meta-analysis of the prevalence of headache

A meta-analysis of the prevalence of gastrointestinal disorders

A meta-analysis of the prevalence of sore throat

Diagnostic aspects

Three reviews described methodologies, protocols, and tools used for establishing the diagnosis of COVID-19 [ 26 , 34 , 38 ]. The use of respiratory swabs (nasal or pharyngeal) or blood specimens to assess the presence of SARS-CoV-2 nucleic acid using RT-PCR assays was the most commonly used diagnostic method mentioned in the included studies. These diagnostic tests have been widely used, but their precise sensitivity and specificity remain unknown. One review included a Chinese study with clinical diagnosis with no confirmation of SARS-CoV-2 infection (patients were diagnosed with COVID-19 if they presented with at least two symptoms suggestive of COVID-19, together with laboratory and chest radiography abnormalities) [ 34 ].

Therapeutic possibilities

Pharmacological and non-pharmacological interventions (supportive therapies) used in treating patients with COVID-19 were reported in five reviews [ 25 , 27 , 34 , 35 , 38 ]. Antivirals used empirically for COVID-19 treatment were reported in seven reviews [ 25 , 27 , 34 , 35 , 37 , 38 , 41 ]; most commonly used were protease inhibitors (lopinavir, ritonavir, darunavir), nucleoside reverse transcriptase inhibitor (tenofovir), nucleotide analogs (remdesivir, galidesivir, ganciclovir), and neuraminidase inhibitors (oseltamivir). Umifenovir, a membrane fusion inhibitor, was investigated in two studies [ 25 , 35 ]. Possible supportive interventions analyzed were different types of oxygen supplementation and breathing support (invasive or non-invasive ventilation) [ 25 ]. The use of antibiotics, both empirically and to treat secondary pneumonia, was reported in six studies [ 25 , 26 , 27 , 34 , 35 , 38 ]. One review specifically assessed evidence on the efficacy and safety of the anti-malaria drug chloroquine [ 27 ]. It identified 23 ongoing trials investigating the potential of chloroquine as a therapeutic option for COVID-19, but no verifiable clinical outcomes data. The use of mesenchymal stem cells, antifungals, and glucocorticoids were described in four reviews [ 25 , 34 , 35 , 38 ].

Laboratory and radiological findings

Of the 18 reviews included in this overview, eight analyzed laboratory parameters in patients with COVID-19 [ 25 , 29 , 30 , 32 , 33 , 34 , 35 , 39 ]; elevated C-reactive protein levels, associated with lymphocytopenia, elevated lactate dehydrogenase, as well as slightly elevated aspartate and alanine aminotransferase (AST, ALT) were commonly described in those eight reviews. Lippi et al. assessed cardiac troponin I (cTnI) [ 25 ], procalcitonin [ 32 ], and platelet count [ 33 ] in COVID-19 patients. Elevated levels of procalcitonin [ 32 ] and cTnI [ 30 ] were more likely to be associated with a severe disease course (requiring intensive care unit admission and intubation). Furthermore, thrombocytopenia was frequently observed in patients with complicated COVID-19 infections [ 33 ].

Chest imaging (chest radiography and/or computed tomography) features were assessed in six reviews, all of which described a frequent pattern of local or bilateral multilobar ground-glass opacity [ 25 , 34 , 35 , 39 , 40 , 41 ]. Those six reviews showed that septal thickening, bronchiectasis, pleural and cardiac effusions, halo signs, and pneumothorax were observed in patients suffering from COVID-19.

Quality of evidence in individual systematic reviews

Table 3 shows the detailed results of the quality assessment of 18 systematic reviews, including the assessment of individual items and summary assessment. A detailed explanation for each decision in each review is available in Additional file 5 .

Using AMSTAR 2 criteria, confidence in the results of all 18 reviews was rated as “critically low” (Table 3 ). Common methodological drawbacks were: omission of prospective protocol submission or publication; use of inappropriate search strategy: lack of independent and dual literature screening and data-extraction (or methodology unclear); absence of an explanation for heterogeneity among the studies included; lack of reasons for study exclusion (or rationale unclear).

Risk of bias assessment, based on a reported methodological tool, and quality of evidence appraisal, in line with the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) method, were reported only in one review [ 25 ]. Five reviews presented a table summarizing bias, using various risk of bias tools [ 25 , 29 , 39 , 40 , 41 ]. One review analyzed “study quality” [ 37 ]. One review mentioned the risk of bias assessment in the methodology but did not provide any related analysis [ 28 ].

This overview of systematic reviews analyzed the first 18 systematic reviews published after the onset of the COVID-19 pandemic, up to March 24, 2020, with primary studies involving more than 60,000 patients. Using AMSTAR-2, we judged that our confidence in all those reviews was “critically low”. Ten reviews included meta-analyses. The reviews presented data on clinical manifestations, laboratory and radiological findings, and interventions. We found no systematic reviews on the utility of diagnostic tests.

Symptoms were reported in seven reviews; most of the patients had a fever, cough, dyspnea, myalgia or muscle fatigue, and gastrointestinal disorders such as diarrhea, nausea, or vomiting. Olfactory dysfunction (anosmia or dysosmia) has been described in patients infected with COVID-19 [ 43 ]; however, this was not reported in any of the reviews included in this overview. During the SARS outbreak in 2002, there were reports of impairment of the sense of smell associated with the disease [ 44 , 45 ].

The reported mortality rates ranged from 0.3 to 14% in the included reviews. Mortality estimates are influenced by the transmissibility rate (basic reproduction number), availability of diagnostic tools, notification policies, asymptomatic presentations of the disease, resources for disease prevention and control, and treatment facilities; variability in the mortality rate fits the pattern of emerging infectious diseases [ 46 ]. Furthermore, the reported cases did not consider asymptomatic cases, mild cases where individuals have not sought medical treatment, and the fact that many countries had limited access to diagnostic tests or have implemented testing policies later than the others. Considering the lack of reviews assessing diagnostic testing (sensitivity, specificity, and predictive values of RT-PCT or immunoglobulin tests), and the preponderance of studies that assessed only symptomatic individuals, considerable imprecision around the calculated mortality rates existed in the early stage of the COVID-19 pandemic.

Few reviews included treatment data. Those reviews described studies considered to be at a very low level of evidence: usually small, retrospective studies with very heterogeneous populations. Seven reviews analyzed laboratory parameters; those reviews could have been useful for clinicians who attend patients suspected of COVID-19 in emergency services worldwide, such as assessing which patients need to be reassessed more frequently.

All systematic reviews scored poorly on the AMSTAR 2 critical appraisal tool for systematic reviews. Most of the original studies included in the reviews were case series and case reports, impacting the quality of evidence. Such evidence has major implications for clinical practice and the use of these reviews in evidence-based practice and policy. Clinicians, patients, and policymakers can only have the highest confidence in systematic review findings if high-quality systematic review methodologies are employed. The urgent need for information during a pandemic does not justify poor quality reporting.

We acknowledge that there are numerous challenges associated with analyzing COVID-19 data during a pandemic [ 47 ]. High-quality evidence syntheses are needed for decision-making, but each type of evidence syntheses is associated with its inherent challenges.

The creation of classic systematic reviews requires considerable time and effort; with massive research output, they quickly become outdated, and preparing updated versions also requires considerable time. A recent study showed that updates of non-Cochrane systematic reviews are published a median of 5 years after the publication of the previous version [ 48 ].

Authors may register a review and then abandon it [ 49 ], but the existence of a public record that is not updated may lead other authors to believe that the review is still ongoing. A quarter of Cochrane review protocols remains unpublished as completed systematic reviews 8 years after protocol publication [ 50 ].

Rapid reviews can be used to summarize the evidence, but they involve methodological sacrifices and simplifications to produce information promptly, with inconsistent methodological approaches [ 51 ]. However, rapid reviews are justified in times of public health emergencies, and even Cochrane has resorted to publishing rapid reviews in response to the COVID-19 crisis [ 52 ]. Rapid reviews were eligible for inclusion in this overview, but only one of the 18 reviews included in this study was labeled as a rapid review.

Ideally, COVID-19 evidence would be continually summarized in a series of high-quality living systematic reviews, types of evidence synthesis defined as “ a systematic review which is continually updated, incorporating relevant new evidence as it becomes available ” [ 53 ]. However, conducting living systematic reviews requires considerable resources, calling into question the sustainability of such evidence synthesis over long periods [ 54 ].

Research reports about COVID-19 will contribute to research waste if they are poorly designed, poorly reported, or simply not necessary. In principle, systematic reviews should help reduce research waste as they usually provide recommendations for further research that is needed or may advise that sufficient evidence exists on a particular topic [ 55 ]. However, systematic reviews can also contribute to growing research waste when they are not needed, or poorly conducted and reported. Our present study clearly shows that most of the systematic reviews that were published early on in the COVID-19 pandemic could be categorized as research waste, as our confidence in their results is critically low.

Our study has some limitations. One is that for AMSTAR 2 assessment we relied on information available in publications; we did not attempt to contact study authors for clarifications or additional data. In three reviews, the methodological quality appraisal was challenging because they were published as letters, or labeled as rapid communications. As a result, various details about their review process were not included, leading to AMSTAR 2 questions being answered as “not reported”, resulting in low confidence scores. Full manuscripts might have provided additional information that could have led to higher confidence in the results. In other words, low scores could reflect incomplete reporting, not necessarily low-quality review methods. To make their review available more rapidly and more concisely, the authors may have omitted methodological details. A general issue during a crisis is that speed and completeness must be balanced. However, maintaining high standards requires proper resourcing and commitment to ensure that the users of systematic reviews can have high confidence in the results.

Furthermore, we used adjusted AMSTAR 2 scoring, as the tool was designed for critical appraisal of reviews of interventions. Some reviews may have received lower scores than actually warranted in spite of these adjustments.

Another limitation of our study may be the inclusion of multiple overlapping reviews, as some included reviews included the same primary studies. According to the Cochrane Handbook, including overlapping reviews may be appropriate when the review’s aim is “ to present and describe the current body of systematic review evidence on a topic ” [ 12 ], which was our aim. To avoid bias with summarizing evidence from overlapping reviews, we presented the forest plots without summary estimates. The forest plots serve to inform readers about the effect sizes for outcomes that were reported in each review.

Several authors from this study have contributed to one of the reviews identified [ 25 ]. To reduce the risk of any bias, two authors who did not co-author the review in question initially assessed its quality and limitations.

Finally, we note that the systematic reviews included in our overview may have had issues that our analysis did not identify because we did not analyze their primary studies to verify the accuracy of the data and information they presented. We give two examples to substantiate this possibility. Lovato et al. wrote a commentary on the review of Sun et al. [ 41 ], in which they criticized the authors’ conclusion that sore throat is rare in COVID-19 patients [ 56 ]. Lovato et al. highlighted that multiple studies included in Sun et al. did not accurately describe participants’ clinical presentations, warning that only three studies clearly reported data on sore throat [ 56 ].

In another example, Leung [ 57 ] warned about the review of Li, L.Q. et al. [ 29 ]: “ it is possible that this statistic was computed using overlapped samples, therefore some patients were double counted ”. Li et al. responded to Leung that it is uncertain whether the data overlapped, as they used data from published articles and did not have access to the original data; they also reported that they requested original data and that they plan to re-do their analyses once they receive them; they also urged readers to treat the data with caution [ 58 ]. This points to the evolving nature of evidence during a crisis.

Our study’s strength is that this overview adds to the current knowledge by providing a comprehensive summary of all the evidence synthesis about COVID-19 available early after the onset of the pandemic. This overview followed strict methodological criteria, including a comprehensive and sensitive search strategy and a standard tool for methodological appraisal of systematic reviews.

In conclusion, in this overview of systematic reviews, we analyzed evidence from the first 18 systematic reviews that were published after the emergence of COVID-19. However, confidence in the results of all the reviews was “critically low”. Thus, systematic reviews that were published early on in the pandemic could be categorized as research waste. Even during public health emergencies, studies and systematic reviews should adhere to established methodological standards to provide patients, clinicians, and decision-makers trustworthy evidence.

Availability of data and materials

All data collected and analyzed within this study are available from the corresponding author on reasonable request.

World Health Organization. Timeline - COVID-19: Available at: https://www.who.int/news/item/29-06-2020-covidtimeline . Accessed 1 June 2021.

COVID-19 Dashboard by the Center for Systems Science and Engineering (CSSE) at Johns Hopkins University (JHU). Available at: https://coronavirus.jhu.edu/map.html . Accessed 1 June 2021.

Anzai A, Kobayashi T, Linton NM, Kinoshita R, Hayashi K, Suzuki A, et al. Assessing the Impact of Reduced Travel on Exportation Dynamics of Novel Coronavirus Infection (COVID-19). J Clin Med. 2020;9(2):601.

Chinazzi M, Davis JT, Ajelli M, Gioannini C, Litvinova M, Merler S, et al. The effect of travel restrictions on the spread of the 2019 novel coronavirus (COVID-19) outbreak. Science. 2020;368(6489):395–400. https://doi.org/10.1126/science.aba9757 .

Article   CAS   PubMed   PubMed Central   Google Scholar  

Fidahic M, Nujic D, Runjic R, Civljak M, Markotic F, Lovric Makaric Z, et al. Research methodology and characteristics of journal articles with original data, preprint articles and registered clinical trial protocols about COVID-19. BMC Med Res Methodol. 2020;20(1):161. https://doi.org/10.1186/s12874-020-01047-2 .

EPPI Centre . COVID-19: a living systematic map of the evidence. Available at: http://eppi.ioe.ac.uk/cms/Projects/DepartmentofHealthandSocialCare/Publishedreviews/COVID-19Livingsystematicmapoftheevidence/tabid/3765/Default.aspx . Accessed 1 June 2021.

NCBI SARS-CoV-2 Resources. Available at: https://www.ncbi.nlm.nih.gov/sars-cov-2/ . Accessed 1 June 2021.

Gustot T. Quality and reproducibility during the COVID-19 pandemic. JHEP Rep. 2020;2(4):100141. https://doi.org/10.1016/j.jhepr.2020.100141 .

Article   PubMed   PubMed Central   Google Scholar  

Kodvanj, I., et al., Publishing of COVID-19 Preprints in Peer-reviewed Journals, Preprinting Trends, Public Discussion and Quality Issues. Preprint article. bioRxiv 2020.11.23.394577; doi: https://doi.org/10.1101/2020.11.23.394577 .

Dobler CC. Poor quality research and clinical practice during COVID-19. Breathe (Sheff). 2020;16(2):200112. https://doi.org/10.1183/20734735.0112-2020 .

Article   Google Scholar  

Bastian H, Glasziou P, Chalmers I. Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med. 2010;7(9):e1000326. https://doi.org/10.1371/journal.pmed.1000326 .

Lunny C, Brennan SE, McDonald S, McKenzie JE. Toward a comprehensive evidence map of overview of systematic review methods: paper 1-purpose, eligibility, search and data extraction. Syst Rev. 2017;6(1):231. https://doi.org/10.1186/s13643-017-0617-1 .

Pollock M, Fernandes RM, Becker LA, Pieper D, Hartling L. Chapter V: Overviews of Reviews. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.1 (updated September 2020). Cochrane. 2020. Available from www.training.cochrane.org/handbook .

Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al. Cochrane handbook for systematic reviews of interventions version 6.1 (updated September 2020). Cochrane. 2020; Available from www.training.cochrane.org/handbook .

Pollock M, Fernandes RM, Newton AS, Scott SD, Hartling L. The impact of different inclusion decisions on the comprehensiveness and complexity of overviews of reviews of healthcare interventions. Syst Rev. 2019;8(1):18. https://doi.org/10.1186/s13643-018-0914-3 .

Pollock M, Fernandes RM, Newton AS, Scott SD, Hartling L. A decision tool to help researchers make decisions about including systematic reviews in overviews of reviews of healthcare interventions. Syst Rev. 2019;8(1):29. https://doi.org/10.1186/s13643-018-0768-8 .

Hunt H, Pollock A, Campbell P, Estcourt L, Brunton G. An introduction to overviews of reviews: planning a relevant research question and objective for an overview. Syst Rev. 2018;7(1):39. https://doi.org/10.1186/s13643-018-0695-8 .

Pollock M, Fernandes RM, Pieper D, Tricco AC, Gates M, Gates A, et al. Preferred reporting items for overviews of reviews (PRIOR): a protocol for development of a reporting guideline for overviews of reviews of healthcare interventions. Syst Rev. 2019;8(1):335. https://doi.org/10.1186/s13643-019-1252-9 .

Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Open Med. 2009;3(3):e123–30.

Krnic Martinic M, Pieper D, Glatt A, Puljak L. Definition of a systematic review used in overviews of systematic reviews, meta-epidemiological studies and textbooks. BMC Med Res Methodol. 2019;19(1):203. https://doi.org/10.1186/s12874-019-0855-0 .

Puljak L. If there is only one author or only one database was searched, a study should not be called a systematic review. J Clin Epidemiol. 2017;91:4–5. https://doi.org/10.1016/j.jclinepi.2017.08.002 .

Article   PubMed   Google Scholar  

Gates M, Gates A, Guitard S, Pollock M, Hartling L. Guidance for overviews of reviews continues to accumulate, but important challenges remain: a scoping review. Syst Rev. 2020;9(1):254. https://doi.org/10.1186/s13643-020-01509-0 .

Covidence - systematic review software. Available at: https://www.covidence.org/ . Accessed 1 June 2021.

Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. 2017;358:j4008.

Borges do Nascimento IJ, et al. Novel Coronavirus Infection (COVID-19) in Humans: A Scoping Review and Meta-Analysis. J Clin Med. 2020;9(4):941.

Article   PubMed Central   Google Scholar  

Adhikari SP, Meng S, Wu YJ, Mao YP, Ye RX, Wang QZ, et al. Epidemiology, causes, clinical manifestation and diagnosis, prevention and control of coronavirus disease (COVID-19) during the early outbreak period: a scoping review. Infect Dis Poverty. 2020;9(1):29. https://doi.org/10.1186/s40249-020-00646-x .

Cortegiani A, Ingoglia G, Ippolito M, Giarratano A, Einav S. A systematic review on the efficacy and safety of chloroquine for the treatment of COVID-19. J Crit Care. 2020;57:279–83. https://doi.org/10.1016/j.jcrc.2020.03.005 .

Li B, Yang J, Zhao F, Zhi L, Wang X, Liu L, et al. Prevalence and impact of cardiovascular metabolic diseases on COVID-19 in China. Clin Res Cardiol. 2020;109(5):531–8. https://doi.org/10.1007/s00392-020-01626-9 .

Article   CAS   PubMed   Google Scholar  

Li LQ, Huang T, Wang YQ, Wang ZP, Liang Y, Huang TB, et al. COVID-19 patients’ clinical characteristics, discharge rate, and fatality rate of meta-analysis. J Med Virol. 2020;92(6):577–83. https://doi.org/10.1002/jmv.25757 .

Lippi G, Lavie CJ, Sanchis-Gomar F. Cardiac troponin I in patients with coronavirus disease 2019 (COVID-19): evidence from a meta-analysis. Prog Cardiovasc Dis. 2020;63(3):390–1. https://doi.org/10.1016/j.pcad.2020.03.001 .

Lippi G, Henry BM. Active smoking is not associated with severity of coronavirus disease 2019 (COVID-19). Eur J Intern Med. 2020;75:107–8. https://doi.org/10.1016/j.ejim.2020.03.014 .

Lippi G, Plebani M. Procalcitonin in patients with severe coronavirus disease 2019 (COVID-19): a meta-analysis. Clin Chim Acta. 2020;505:190–1. https://doi.org/10.1016/j.cca.2020.03.004 .

Lippi G, Plebani M, Henry BM. Thrombocytopenia is associated with severe coronavirus disease 2019 (COVID-19) infections: a meta-analysis. Clin Chim Acta. 2020;506:145–8. https://doi.org/10.1016/j.cca.2020.03.022 .

Ludvigsson JF. Systematic review of COVID-19 in children shows milder cases and a better prognosis than adults. Acta Paediatr. 2020;109(6):1088–95. https://doi.org/10.1111/apa.15270 .

Lupia T, Scabini S, Mornese Pinna S, di Perri G, de Rosa FG, Corcione S. 2019 novel coronavirus (2019-nCoV) outbreak: a new challenge. J Glob Antimicrob Resist. 2020;21:22–7. https://doi.org/10.1016/j.jgar.2020.02.021 .

Marasinghe, K.M., A systematic review investigating the effectiveness of face mask use in limiting the spread of COVID-19 among medically not diagnosed individuals: shedding light on current recommendations provided to individuals not medically diagnosed with COVID-19. Research Square. Preprint article. doi : https://doi.org/10.21203/rs.3.rs-16701/v1 . 2020 .

Mullins E, Evans D, Viner RM, O’Brien P, Morris E. Coronavirus in pregnancy and delivery: rapid review. Ultrasound Obstet Gynecol. 2020;55(5):586–92. https://doi.org/10.1002/uog.22014 .

Pang J, Wang MX, Ang IYH, Tan SHX, Lewis RF, Chen JIP, et al. Potential Rapid Diagnostics, Vaccine and Therapeutics for 2019 Novel coronavirus (2019-nCoV): a systematic review. J Clin Med. 2020;9(3):623.

Rodriguez-Morales AJ, Cardona-Ospina JA, Gutiérrez-Ocampo E, Villamizar-Peña R, Holguin-Rivera Y, Escalera-Antezana JP, et al. Clinical, laboratory and imaging features of COVID-19: a systematic review and meta-analysis. Travel Med Infect Dis. 2020;34:101623. https://doi.org/10.1016/j.tmaid.2020.101623 .

Salehi S, Abedi A, Balakrishnan S, Gholamrezanezhad A. Coronavirus disease 2019 (COVID-19): a systematic review of imaging findings in 919 patients. AJR Am J Roentgenol. 2020;215(1):87–93. https://doi.org/10.2214/AJR.20.23034 .

Sun P, Qie S, Liu Z, Ren J, Li K, Xi J. Clinical characteristics of hospitalized patients with SARS-CoV-2 infection: a single arm meta-analysis. J Med Virol. 2020;92(6):612–7. https://doi.org/10.1002/jmv.25735 .

Yang J, Zheng Y, Gou X, Pu K, Chen Z, Guo Q, et al. Prevalence of comorbidities and its effects in patients infected with SARS-CoV-2: a systematic review and meta-analysis. Int J Infect Dis. 2020;94:91–5. https://doi.org/10.1016/j.ijid.2020.03.017 .

Bassetti M, Vena A, Giacobbe DR. The novel Chinese coronavirus (2019-nCoV) infections: challenges for fighting the storm. Eur J Clin Investig. 2020;50(3):e13209. https://doi.org/10.1111/eci.13209 .

Article   CAS   Google Scholar  

Hwang CS. Olfactory neuropathy in severe acute respiratory syndrome: report of a case. Acta Neurol Taiwanica. 2006;15(1):26–8.

Google Scholar  

Suzuki M, Saito K, Min WP, Vladau C, Toida K, Itoh H, et al. Identification of viruses in patients with postviral olfactory dysfunction. Laryngoscope. 2007;117(2):272–7. https://doi.org/10.1097/01.mlg.0000249922.37381.1e .

Rajgor DD, Lee MH, Archuleta S, Bagdasarian N, Quek SC. The many estimates of the COVID-19 case fatality rate. Lancet Infect Dis. 2020;20(7):776–7. https://doi.org/10.1016/S1473-3099(20)30244-9 .

Wolkewitz M, Puljak L. Methodological challenges of analysing COVID-19 data during the pandemic. BMC Med Res Methodol. 2020;20(1):81. https://doi.org/10.1186/s12874-020-00972-6 .

Rombey T, Lochner V, Puljak L, Könsgen N, Mathes T, Pieper D. Epidemiology and reporting characteristics of non-Cochrane updates of systematic reviews: a cross-sectional study. Res Synth Methods. 2020;11(3):471–83. https://doi.org/10.1002/jrsm.1409 .

Runjic E, Rombey T, Pieper D, Puljak L. Half of systematic reviews about pain registered in PROSPERO were not published and the majority had inaccurate status. J Clin Epidemiol. 2019;116:114–21. https://doi.org/10.1016/j.jclinepi.2019.08.010 .

Runjic E, Behmen D, Pieper D, Mathes T, Tricco AC, Moher D, et al. Following Cochrane review protocols to completion 10 years later: a retrospective cohort study and author survey. J Clin Epidemiol. 2019;111:41–8. https://doi.org/10.1016/j.jclinepi.2019.03.006 .

Tricco AC, Antony J, Zarin W, Strifler L, Ghassemi M, Ivory J, et al. A scoping review of rapid review methods. BMC Med. 2015;13(1):224. https://doi.org/10.1186/s12916-015-0465-6 .

COVID-19 Rapid Reviews: Cochrane’s response so far. Available at: https://training.cochrane.org/resource/covid-19-rapid-reviews-cochrane-response-so-far . Accessed 1 June 2021.

Cochrane. Living systematic reviews. Available at: https://community.cochrane.org/review-production/production-resources/living-systematic-reviews . Accessed 1 June 2021.

Millard T, Synnot A, Elliott J, Green S, McDonald S, Turner T. Feasibility and acceptability of living systematic reviews: results from a mixed-methods evaluation. Syst Rev. 2019;8(1):325. https://doi.org/10.1186/s13643-019-1248-5 .

Babic A, Poklepovic Pericic T, Pieper D, Puljak L. How to decide whether a systematic review is stable and not in need of updating: analysis of Cochrane reviews. Res Synth Methods. 2020;11(6):884–90. https://doi.org/10.1002/jrsm.1451 .

Lovato A, Rossettini G, de Filippis C. Sore throat in COVID-19: comment on “clinical characteristics of hospitalized patients with SARS-CoV-2 infection: a single arm meta-analysis”. J Med Virol. 2020;92(7):714–5. https://doi.org/10.1002/jmv.25815 .

Leung C. Comment on Li et al: COVID-19 patients’ clinical characteristics, discharge rate, and fatality rate of meta-analysis. J Med Virol. 2020;92(9):1431–2. https://doi.org/10.1002/jmv.25912 .

Li LQ, Huang T, Wang YQ, Wang ZP, Liang Y, Huang TB, et al. Response to Char’s comment: comment on Li et al: COVID-19 patients’ clinical characteristics, discharge rate, and fatality rate of meta-analysis. J Med Virol. 2020;92(9):1433. https://doi.org/10.1002/jmv.25924 .

Download references

Acknowledgments

We thank Catherine Henderson DPhil from Swanscoe Communications for pro bono medical writing and editing support. We acknowledge support from the Covidence Team, specifically Anneliese Arno. We thank the whole International Network of Coronavirus Disease 2019 (InterNetCOVID-19) for their commitment and involvement. Members of the InterNetCOVID-19 are listed in Additional file 6 . We thank Pavel Cerny and Roger Crosthwaite for guiding the team supervisor (IJBN) on human resources management.

This research received no external funding.

Author information

Authors and affiliations.

University Hospital and School of Medicine, Universidade Federal de Minas Gerais, Belo Horizonte, Minas Gerais, Brazil

Israel Júnior Borges do Nascimento & Milena Soriano Marcolino

Medical College of Wisconsin, Milwaukee, WI, USA

Israel Júnior Borges do Nascimento

Helene Fuld Health Trust National Institute for Evidence-based Practice in Nursing and Healthcare, College of Nursing, The Ohio State University, Columbus, OH, USA

Dónal P. O’Mathúna

School of Nursing, Psychotherapy and Community Health, Dublin City University, Dublin, Ireland

Department of Anesthesiology, Intensive Care and Pain Medicine, University of Münster, Münster, Germany

Thilo Caspar von Groote

Department of Sport and Health Science, Technische Universität München, Munich, Germany

Hebatullah Mohamed Abdulazeem

School of Health Sciences, Faculty of Health and Medicine, The University of Newcastle, Callaghan, Australia

Ishanka Weerasekara

Department of Physiotherapy, Faculty of Allied Health Sciences, University of Peradeniya, Peradeniya, Sri Lanka

Cochrane Croatia, University of Split, School of Medicine, Split, Croatia

Ana Marusic, Irena Zakarija-Grkovic & Tina Poklepovic Pericic

Center for Evidence-Based Medicine and Health Care, Catholic University of Croatia, Ilica 242, 10000, Zagreb, Croatia

Livia Puljak

Cochrane Brazil, Evidence-Based Health Program, Universidade Federal de São Paulo, São Paulo, Brazil

Vinicius Tassoni Civile & Alvaro Nagib Atallah

Yorkville University, Fredericton, New Brunswick, Canada

Santino Filoso

Laboratory for Industrial and Applied Mathematics (LIAM), Department of Mathematics and Statistics, York University, Toronto, Ontario, Canada

Nicola Luigi Bragazzi

You can also search for this author in PubMed   Google Scholar

Contributions

IJBN conceived the research idea and worked as a project coordinator. DPOM, TCVG, HMA, IW, AM, LP, VTC, IZG, TPP, ANA, SF, NLB and MSM were involved in data curation, formal analysis, investigation, methodology, and initial draft writing. All authors revised the manuscript critically for the content. The author(s) read and approved the final manuscript.

Corresponding author

Correspondence to Livia Puljak .

Ethics declarations

Ethics approval and consent to participate.

Not required as data was based on published studies.

Consent for publication

Not applicable.

Competing interests

The authors declare no conflict of interest.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: appendix 1..

Search strategies used in the study.

Additional file 2: Appendix 2.

Adjusted scoring of AMSTAR 2 used in this study for systematic reviews of studies that did not analyze interventions.

Additional file 3: Appendix 3.

List of excluded studies, with reasons.

Additional file 4: Appendix 4.

Table of overlapping studies, containing the list of primary studies included, their visual overlap in individual systematic reviews, and the number in how many reviews each primary study was included.

Additional file 5: Appendix 5.

A detailed explanation of AMSTAR scoring for each item in each review.

Additional file 6: Appendix 6.

List of members and affiliates of International Network of Coronavirus Disease 2019 (InterNetCOVID-19).

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Borges do Nascimento, I.J., O’Mathúna, D.P., von Groote, T.C. et al. Coronavirus disease (COVID-19) pandemic: an overview of systematic reviews. BMC Infect Dis 21 , 525 (2021). https://doi.org/10.1186/s12879-021-06214-4

Download citation

Received : 12 April 2020

Accepted : 19 May 2021

Published : 04 June 2021

DOI : https://doi.org/10.1186/s12879-021-06214-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Coronavirus
• Evidence-based medicine
• Infectious diseases

BMC Infectious Diseases

ISSN: 1471-2334

• General enquiries: [email protected]