Zhu Liduzi Jiesisibieke
Wen-Yi Liu
Yu-Pei Yang5
Tao-Hsin Tung- 1Evidence-Based Medicine Center, Taizhou Hospital of Zhejiang Province Affiliated to Wenzhou Medical University, Linhai, Zhejiang, China
- 2Institute for Hospital Management, Tsinghua University, Shenzhen, China
- 3Department of Health Policy Management, Bloomberg School of Public Health, Johns Hopkins University, Baltimore, MD, United States
- 4Shanghai Bluecross Medical Science Institute, Shanghai, China
- 5Department of Hematology, Taizhou Hospital of Zhejiang Province Affiliated to Wenzhou Medical University, Linhai, Zhejiang, China
Objectives: This umbrella meta-analysis aims to provide comprehensive and synthesized evidence regarding the effectiveness and safety of COVID-19 vaccinations based on current studies.
Methods: Studies from the Cochrane Library, PubMed, and EMBASE, published before 10 December 2021, were included in the analysis. The pooled results of effectiveness and safety were estimated and shown in forest plots.
Results: We included nineteen studies (fifteen studies regarding safety and nine regarding effectiveness) in the analysis. The mRNA vaccines, adenovirus vector vaccines, subunit vaccines, and inactivated vaccines were found to be effective; however, mRNA vaccines, adenovirus vector vaccines and subunit vaccines were associated with local adverse events and systemic events when compared with inactivated vaccines.
Conclusion: Our study suggested that till date, COVID-19 vaccination is still a preferred pharmaceutical way to control the widespread pandemic. However, all reported adverse events should be revisited to provide further evidence for mass vaccinations.
Introduction
Pandemics and epidemics have devastated human societies in the course of history [1], and COVID-19 is a classic example. According to the data from WHO Coronavirus (COVID-19) Dashboard, as of 13 January 2022, there have been 315,345,967 confirmed cases of COVID-19, including 5,510,174 deaths [2]. Previous results indicated that not only countries highly dependent on foreign trade are more negatively affected, but also on average, each additional month of crisis costs 2.5-3% of global GDP [3]. Before the COVID-19 vaccination was invented, non-pharmaceutical interventions were used worldwide as the only option to delay and moderate the spread of COVID-19 [4]. As of 13 January 2022, a total of 9,194,549,698 vaccine doses have been administered [2]. Despite the large numbers of vaccinations, preparations for the public to accept the vaccination must be carefully launched worldwide as COVID-19 vaccination hesitancy is still common [5]. The attitudes toward COVID-19 vaccination varies in different countries and regions. A large national study among adults in the US showed that approximately 22% of the participants reported vaccination hesitancy [6]. Contrastingly, the Chinese population presented a higher acceptance and positive attitudes toward COVID-19 vaccination [7]. Multiple factors contribute to COVID-19 vaccination hesitancy, such as concerns about the vaccine efficacy, safety, side effects, convenience, price, financial motivation of the authorities and pharmaceutical companies, and beliefs regarding its necessity, insufficient testing, and quick pace of its development. However, the primary reason among these factors is the fear of safety and the side effects [8]. Presently, multiple COVID-19 vaccinations have been approved globally, such as mRNA vaccination, inactivated vaccination, protein subunit vaccination, replicating/non-replicating viral vectors and others. Each of these has its own advantages and disadvantages regarding effectiveness and safety [9]. There are many meta-analysis and systematic reviews investigating the effectiveness and safety of various COVID-19 vaccinations. Regarding vaccination effectiveness, some reviews summarize it by means of antibody levels [10] and others evaluate it based on the prevention of COVID-19 [11]. With respect to safety, some studies divide the side effects to systemic adverse events (including fatigue, vomiting, fever, myalgia, and diarrhea) and local adverse events (injection site pain, itching, swelling, and redness) [12, 13], while others look at other rare situations such as thromboembolic events, myocarditis/pericarditis events, or allergic reactions [14, 15]. Umbrella reviews systematically collect systematic reviews and meta-analyses, and umbrella reviews have often been used to provide a comprehensive and holistic understanding about an issue with a broader range of studies, and it is also time-efficient to conduct. Therefore, it is believed to have the possibility to provide the highest level of evidence [16, 17]. However, till date, there has been no umbrella meta-analysis to summarize the effectiveness and safety of different types of COVID-19 vaccinations.
Given the significance of this global pandemic and numerous reviews about the effectiveness and safety of COVID-19 vaccinations and variations among the findings, we conducted this umbrella meta-analysis to thoroughly address the following issues: 1) the effectiveness of mRNA vaccines (i.e., mRNA-1273, BNT162b1, BNT161b2, etc.), adenovirus vector vaccines (i.e., Ad6.COV2·S, non-replicating viral vector vaccination, etc.), subunit vaccines (i.e., protein subunit vaccination, NVX-CoV2373, etc.), and inactivated vaccines (i.e., CoronaVac, BBIBP-CorV, etc.) and 2) the safety of these four types of vaccinations and the outcomes including systemic, local, and other reactions. We believe this umbrella meta-analysis would give a comprehensive understanding of the COVID-19 vaccination and be instrumental in mitigating the vaccination hesitancy to a certain extent.
Methods
Search Strategy and Selection Criteria
References for this review were identified through searches of the Cochrane Library, PubMed, and EMBASE for related studies from onset to 10 December 2021. Since we intended to include all the related studies, instead of confining the key words to “effectiveness” or “safety,” the search string used “COVID-19,” “COVID-19,” “SARS-CoV-2,” “severe acute respiratory syndrome coronavirus 2,” “Corona Virus Disease 2019,” “vaccine,” “vaccination,” “systematic review” and “meta-analysis,” with no restrictions on language. The search string was “(COVID-19 OR COVID-19 OR SARS-CoV-2 OR severe acute respiratory syndrome coronavirus 2 OR Corona Virus Disease 2019) AND (vaccine OR vaccination) AND (Systematic review OR meta-analysis)”. We also checked the references of selected studies and other relevant sources to further identify related studies. The search strategy was performed in conformity with Preferred Reporting Items for Systematic Reviews and Meta-Analyses 2020 guidelines (PRISMA 2020).
Study Selection
The inclusion criteria for the studies were: 1) meta-analysis; 2) the outcome of the study should relate to vaccine effectiveness, safety, and efficacy; 3) the study should show the pooled result (i.e., pooled odds ratio) and the measurement of prevalence should show the details of every included study for cross-checking and 4) the type of COVID-19 vaccination should be clarified in the study. We excluded studies that did not specify the type of COVID-19 vaccination. Regarding “safety,” our interest was whether the adverse events occurred after the COVID-19 vaccination; hence, adverse events before the injection of COVID-19 vaccination were not included. We scrutinized the titles or abstracts of the search results; the full text was acquired when a study met the inclusion criteria. The full text was investigated to determine the possibility of related data. Two authors (Miss Zhu Liduzi Jiesisibieke and Wen-Yi Liu) selected relevant studies independently; any arising disputes were settled through discussion with the two primary authors (Ching-Wen- Chien and Tao-Hsin Tung).
Data Extraction
We systematically reviewed meta-analysis articles and discussed the effectiveness and safety of COVID-19 vaccinations. Meta-analyses with insufficient data were excluded. We obtained the raw data from eligible meta-analyses and extracted and summarized the information of the first author, year, number of studies included in individual meta-analysis as well as the study purpose, intervention, and outcome. In addition, we extracted the original data set of each study for further meta-analysis. For the measurement of prevalence, we investigated every original research for detailed information.
Statistical Analysis
For binary variables, odds ratios (ORs) and risk ratios (RRs) were used to pool the results from each meta-analysis together, based on the number of cases in vaccination group and control group or the extracted ORs and RRs from each study. For continuous variables (i.e., prevalence of adverse events, etc.), we used mean differences (MDs). The I2 statistic was used to assess the level of statistical heterogeneity, and an I2 value of 50% or more constitutes substantial heterogeneity [18]. Data analyses were performed using The Review Manager 5.4. To provide an evaluation of the quality of the included studies as well as measure the risk of bias of the included studies, we also valued the meta-analyses based on 7 criteria, which is commonly used in umbrella reviews [16, 19].
Results
Literature Search
We obtained 707 articles from the electronic database search (433 from the PubMed, 194 from the Embase, and 82 from the Cochrane library) (Figure 1). No studies from preprint platforms were included, considering the lack of peer-review process. After removing duplicates, the analyses included 19 studies.
FIGURE 1. Preferred Reporting Items for Systematic Reviews and Meta-Analyses flow chart (Global, 2021).
Characteristics of the Included Studies
Among the nineteen included studies, fifteen studies reported on safety [10, 12–15, 20–29] and nine studies reported on effectiveness/efficacy [10–12, 20, 23, 27, 30–32]. Three studies reported the number of participants in both experiment and control group for effectiveness of attenuated COVID-19 vaccinations (i.e., mRNA vaccination, etc.) [12, 20, 27]. Two studies reported the number of participants in both experiment and control group for effectiveness of inactivated COVID-19 vaccinations [12, 20]. One study reported breakthrough infectious cases of attenuated COVID-19 vaccinations [31]. One study reported COVID-19 event rate regardless of COVID-19 vaccination types [32]. One study reported hazard ratio of incidence of COVID-19 regardless of COVID-19 vaccination types [31]. One study reported effectiveness of attenuated COVID-19 vaccinations based on anitbody levels [10]. Three studies reported effectiveness of attenuated COVID-19 vaccinations based on prevention of COVID-19 infections [11, 15, 23]. Two studies reported effectiveness of inactivated COVID-19 vaccinations based on prevention of COVID-19 infections [11, 15].
Three studies reported both local and systemic adverse events associated with attenuated COVID-19 vaccinations [10, 12, 24]. Two studies reported both local and systemic adverse events associated with inactivated COVID-19 vaccinations [10, 24]. One study reported total adverse events associated with attenuated COVID-19 vaccinations [21]. One study reported total adverse events associated with inactivated COVID-19 vaccinations [12]. Two studies reported severe adverse events associated with attenuated COVID-19 vaccinations [20, 24]. One study reported other adverse events (including thromboembolic events and myocarditis/pericarditis events, etc.) associated with attenuated COVID-19 vaccinations [28]. Four studies reported prevalence of adverse events associated with COVID-19 vaccinations [13–15]. The study by McDonald et al. reported total adverse events regardless of the COVID-19 vaccination type [22]. The study by Chen et al. reported headache and myalgia regardless of COVID-19 vaccination type [26]. The studies by Liu et al. and Alhumaid et al. did not include the detailed information of how they concluded the results of the pooled prevalence of adverse events [23, 25]; hence, we did not include these in the results for safety.
COVID-19 Vaccination Effectiveness
Nine of the included studies examined the effectiveness of COVID-19 vaccinations. Cheng et al. included eight studies to investigate the effectiveness of COVID-19 vaccines in phase III trials [12]. Sharif et al. included seven studies to investigate the efficacy of mRNA vaccines and adenovirus vector vaccines; they found that efficacy of mRNA vaccines (85%, 95% CI: 82–88) was higher than adenovirus vector vaccines (73%, 95% CI: 69–77) [27]. Fan et al. investigated the efficacy of mRNA vaccinations, non-replicating viral vector, and inactivated vaccines after one dose and two doses; they found that mRNA vaccinations (OR: 0.05, 95% CI: 0.02 0.13) conferred a lesser risk of COVID-19 than non-replicating viral vector and inactivated vaccines [20]. Kow et al. investigated the effectiveness of mRNA vaccinations (BNT161b2) based on large observational studies; they concluded that after 14 days, the effectiveness of the first dose was 53% (95% CI: 32%–68%) and of the second dose was 96% (95% CI: 95%–97%) [31]. Cai et al. included 17 phase I/II clinical trials and 4 phase III trials; they found that worldwide mRNA had the highest efficacy of 94.29% [15]. Liu et al. found that a single dose of COVID-19 vaccine was 41% effective in preventing COVID-19 infections [23]. Zheng et al. concluded that the effectiveness of Pfizer, Moderna, and CoronaVac vaccinations was 91.2%, 98.1% and 65.7%, respectively [11]. Chandan et al. included the partially vaccinated, fully vaccinated, and unvaccinated health workers; they found that the COVID-19 infection rate was quite low in partially and fully vaccinated health workers [32]. Naranbhai et al. found that the risk of breakthrough infection after BNT161b2 relative to Moderna was 1.53 (95% CI: 1.52–1.55), and after adenovirus vector vaccine was 2.54 (95% CI: 2.52–2.56). Since this was the only study that reported risk ratio (RR) of breakthrough cases, we did not conduct the forest plot to show the pooled RR of COVID-19 breakthrough risk [30]. Since only Promohammad et al.’s study reported the vaccine efficacy based on antibody reactions, and found that mRNA vaccines had 94.6% (95% CI: 0.936–0.954) efficacy in phase II/III RCTs, while adenovirus-vectored vaccines had 80.2% (95% CI: 0.56–0.93) efficacy [10], we did not use forest plot to calculate the pooled result. The pooled RRs of mRNA vaccinations, adenovirus vectored vaccinations, and inactivated vaccinations were 0.12 (95% CI: 0.01–1.07), 0.23 (95% CI: 0.19–0.29) and 0.38 (95% CI: 0.34–0.43), respectively. The pooled HR of mRNA vaccination was 0.27 (95% CI: 0.16–0.46). The pooled vaccination effectiveness of mRNA vaccinations and inactivated vaccinations was 0.95 (95% CI: 0.93–0.98) and 0.66 (95% CI: 0.63–0.68), respectively. The COVID-19 event rate after the overall COVID-19 vaccinations was 0.02 (95% CI: 0.01–0.03) (Figure 2).
FIGURE 2. Forest plots of COVID-19 vaccination effectiveness (Global, 2021). (A) Pooled risk ratio of messenger Ribonucleic acid vaccination. (B) Pooled risk ratio of Adenovirus vector vaccination. (C) Pooled risk ratio of Inactivated vaccination. (D) Pooled hazard ratio of messenger Ribonucleic acid vaccination. (E) Vaccination effectiveness of messenger Ribonucleic acid vaccination. (F) Vaccination effectiveness of inactivated vaccination. (G) COVID-19 event rate after overall COVID-19 vaccination.
COVID-19 Vaccination Safety
Fifteen studies reported on safety of COVID-19 vaccinations. Cheng et al. categorized the adverse events into any adverse events, local adverse events, and systemic adverse events, and found that the overall adverse events showed an increase in the vaccine group after the first or second dose when compared with the control group [12]. Wu et al. identified 87 studies and found that compared with mRNA vaccination group, the inactivated group, protein subunit vaccines, and DNA vaccines had lower local and systemic reactions [24]. Ling et al. included nine studies and found that inactivated vaccines, RNA vaccines, and adenovirus vector vaccines were associated with higher incidence of adverse events [21]. Uaprasert et al. found no increased risk of thromboembolism and hemorrhage after COVID-19 vaccinations [28]. Promohammad et al. included 58,889 cases and found that mRNA vaccinations reported the highest side effects except for diarrhea and arthralgia, while the aluminums-adjuvanted vaccines had the lowest systemic and local side effects except for injection site redness [10]. Cai et al. believed that the adverse events associated with COVID-19 vaccinations were tolerable, but still needed to be identified and addressed in a timely manner [15]. Sathian et al. included 12 studies and considered that solicited and unsolicited systemic adverse events should be addressed with caution [29]. Chen et al. investigated adverse reactions to inoculation doses in various age groups, and believed that COVID-19 vaccinations were acceptable for mass vaccination [13]. Liu et al. investigated the incidence of adverse events, severe adverse events, and deaths related to adverse events, and found that the overall pooled incidence rate was 1.5% (95% CI: 1.4–1.6%) for adverse events, 0.4 per 10,000 (95% CI: 0.2–0.5) for severe adverse events, and 0.1 per 10,000 (0.1–0.2) for death [23]. The study by Greenhawt et al. found that concerns for anaphylaxis might hinder the COVID-19 vaccinations and concluded the incidence of COVID-19 vaccine anaphylaxis as 7.91 cases per million; thus recommending vaccination [14]. Fan et al. found that mRNA was associated with more serious adverse events compared with vector and inactivated vaccines [20]. McDonald et al. found that the pooled risk ratio of incidence of solicited adverse events was 1.74 (95% CI: 1.49–2.02) [22]. In their study, Alhumaid et al. concluded that pooled prevalence estimates of anaphylaxis to Pfizer and Moderna vaccinations were 5.0% (95% CI: 2.9–7.2) [25]. Chen et al. found that headache and myalgia were still common with a rate of 29.5% and 19.2%, respectively in phase III trials [26]. Sharif et al. found that fatigue was the most prevalent local adverse event associated with mRNA vaccinations while fever was most prevalent in adenovirus vector vaccines [27]. Since a better way to conduct the umbrella meta-analysis is to review the original studies to recalculate the prevalence, the studies by Liu et al. and Greenhawt et al. provided information regarding the prevalence of adverse events, but did not provide the information to cross-check; hence, the studies were not included [14, 23]. The pooled RRs of local adverse events, systemic adverse events, and total adverse events associated with mRNA vaccinations were 3.52 (95% CI: 2.85–4.33), 1.32 (95% CI: 1.11–1.50), and 1.99 (95% CI: 1.83–2.18), respectively. The pooled ORs of local adverse events and systemic adverse events associated with mRNA vaccinations were 37.77 (95% CI: 16.45–86.70) and 5.62 (95% CI: 4.88–6.48), respectively.
The pooled RRs of local adverse events and total adverse events associated with adenovirus-vectored vaccinations were 1.89 (95% CI: 1.81–1.97) and 1.55 (95% CI: 1.00–2.39). The pooled ORs of local adverse events and total adverse events associated with adenovirus-vectored vaccinations were 3.55 (95% CI: 1.65–7.64) and 3.05 (95% CI: 2.26–4.14). The pooled ORs of local adverse events and total adverse events associated with subunit vaccinations were 5.34 (95% CI: 1.86–1.29) and 1.44 (95% CI: 0.95–2.17). The pooled ORs of local adverse events and total adverse events associated with inactivated vaccinations were 1.84 (95% CI: 1.00–3.37), and 0.42 (95% CI: 0.25–0.73). The RRs of other adverse events associated with overall COVID-19 vaccinations was 1.18 (95% CI: 0.79–1.77). The prevalence of any adverse events associated with overall COVID-19 was 0.45 (95% CI: 0.12–0.78). Due of the lack of data, we did not conduct subgroup analysis of systemic adverse events associated with adenovirus vector vaccines and any adverse events associated with either subunit vaccinations or inactivated vaccinations (Figure 3).
FIGURE 3. Forest plots of COVID-19 vaccination safety (Global, 2021). (A) The pooled risk ratio of local adverse events associated with messenger Ribonucleic acid vaccination. (B) The pooled risk ratio of systemic adverse events associated with messenger Ribonucleic acid vaccination. (C) The pooled risk ratio of any adverse events associated with messenger Ribonucleic acid vaccination. (D) The pooled risk ratio of local adverse events associated with adenovirus vector vaccination. (E) The pooled risk ratio of any adverse events associated with adenovirus vector vaccination. (F) The pooled odds ratio of systemic adverse events associated with messenger Ribonucleic acid vaccination. (G) The pooled odds ratio of systemic adverse events associated with messenger Ribonucleic acid vaccination. (H) The pooled odds ratio of local adverse events associated with adenovirus vector vaccination. (I) The pooled odds ratio of systemic adverse events associated with adenovirus vector vaccination. (J) The pooled odds ratio of local adverse events associated with subunit vaccination. (K) The pooled odds ratio of systemic adverse events associated with subunit vaccination. (L) The pooled risk ratio of other adverse events associated with overall COVID-19 vaccination. (M) The pooled odds ratio of local adverse events associated with inactivated vaccination. (N) The pooled odds ratio of systemic adverse events associated with inactivated vaccination. (O) The pooled prevalence of ant adverse events associated with overall COVID-19 vaccination.
Evidence Level of the Included Studies
Based on the seven criteria, we have provided the level of evidence in Table 1. Two of the included studies were rated as “highly suggestive,” another two of the studies were rated as “suggestive,” the remaining 15 studies were rated as “weak.”
TABLE 1. Characteristics of the included studies (Global, 2021).
Discussion
Clinical Implications
To the best of our knowledge, our review is the first umbrella meta-analysis investigating the effectiveness and safety of COVID-19 vaccinations, despite many meta-analyses on this topic. The mRNA vaccines, adenovirus vector vaccines, subunit vaccines, and inactivated vaccines were found to be effective. However, regarding safety, mRNA vaccines, adenovirus vector vaccines and subunit vaccines were associated with local adverse events and systemic events when compared with inactivated vaccines.
After the genetic sequences of COVID-19 were accessible, many countries have given top priority to the development of COVID-19 vaccinations. Compared to traditional vaccination platforms, such as inactivated, live attenuated virus, protein subunit and replicating/non-replicating viral vectors, novel platforms such as DNA and mRNA vaccinations, can be quicker, since they require no culture or fermentation; instead they use synthetic processes [33, 34]. Hopefully, in future there will be diverse pharmaceutical ways to not just protect against, but also cure COVID-19, reducing morbidity and mortality [35]. However, before that happens, it is imperative to continue studying COVID-19 vaccination for its effectiveness and safety.
Effectiveness of COVID-19 Vaccinations
Previous studies have investigated the effectiveness of different types of COVID-19 vaccinations. In COVID-19, the spike glycoprotein which plays a significant role in viral infection and pathogenesis, is composed of 672 amino acids [36], and can bind and neutralize antibodies [37]. Different COVID-19 vaccinations induce antibodies, resulting in differing effectiveness [38]. The mRNA vaccines are the most effective vaccinations; however inactivated vaccinations were not as effective as mRNA vaccinations [12]. The reason for this may be that via physical or chemical methods, the inactivated virus loses its virulence to the original target organ and maintains the immunogenicity of its corresponding antigen. Inactivated vaccination has stable conformation-dependent epitope expression and is easy to mass-produce [39]. Since the inactivated virus vaccine only induced antibodies, the immune effect is not as satisfactory [21]. Off late, mRNA vaccination has developed dramatically as its basic mechanism is that the host uses the perfusion-stabilized mRNA to generate the target protein, which leads to the immune response [40]. The effectiveness information of adenovirus vaccination differed in the first and the second dosage, perhaps because it used different adenovirus vectors, resulting in various immune responses [41]. The subunit vaccine carries the protein from pathogenic sequence and activates the immune response [42]. The mutations in COVID-19 also worsen the situation. Studies reported changes in the virus, for example, lB.1.351 variant showed signs of vaccine escape [43]. Hence, which type of COVID-19 vaccination is most effective needs further evidence and time.
Safety of COVID-19 Vaccinations
Adverse events happen when antibodies bind to the targeted virus and then the resulting antibody/virus complex enhances uptake of the virus by host macrophages and other immune cells, causing a systemic vasculitis-like disease [44, 45]. Prevention of severe adverse events has been an important goal for COVID-19 vaccination programs. The mRNA vaccinations were associated with higher adverse events, while the inactivated vaccinations had the lowest adverse events; the adenovirus-vectored vaccine was associated with diarrhea and arthralgia [12, 26]. For subunit vaccines, the most common local adverse event was mild injection site pain while the most common systemic adverse events were headache, fatigue, and myalgia [46]. The reasons for fewer adverse events in inactivated vaccination might be its mechanism, technology, the alum-adjuvanted, or other factors [10]. Although in this study, we did not analyze each specific adverse event, we included systemic adverse events (including fatigue, vomiting, fever, myalgia, and diarrhea) and local adverse events (injection site pain, itch, swelling, and redness). We also included other adverse events, including thromboembolic and allergies. Most of the adverse events were transient and self-limited that would usually resolve with a few days after receiving vaccinations [47]. COVID-19 infection was found to affect immune system, respiratory system, cardiovascular system, and neurological system; it also led to cutaneous and gastrointestinal manifestations and impaired hepatic and renal functions [48]. In such adverse events, COVID-19 vaccinations were found not to increase the risk of arrhythmia, acute kidney injury, pulmonary embolism, deep-vein thrombosis, myocardial infarction, pericarditis, and intracranial haemorrhage [49]. Though dosage is closely related to adverse events, presently no significant association was found between dosage and adverse events [13, 24, 26].
Methodological Considerations
Since this is the first umbrella meta-analysis to investigate the effectiveness and safety of COVID-19 vaccinations, some of its limitations must be considered. First, our results only imply the effectiveness and safety of the mRNA vaccines, adenovirus vector vaccines, subunit vaccines, and inactivated vaccines; we did not compare the effectiveness and safety of these different types of COVID-19 vaccination. Therefore, further studies comparing the effectiveness and safety of these vaccinations are needed. Second, since the studies were conducted in different countries that had varying incidence of COVID-19, heterogeneity was inherent in this review. Another source of heterogeneity were the participants; since we did not conduct subgroup analysis according to different phases, there might be some biases. Third, since the COVID-19 vaccination has been developing at a rapid pace in relatively short time, our investigations in this review were limited to short-term effectiveness and safety. Long-term evidence is still needed in future. In addition, the presence of different vaccination types required a different statistical analysis such a network meta-analysis, however, it is very difficult to do network meta-analysis in this study.
Conclusion
Despite COVID-19 vaccination being the preferred pharmaceutical way to control the pandemic at present, caution should be exercised regarding the reported adverse events to provide further evidence for mass vaccinations. The researches about the effectiveness and safety of COVID-19 vaccinations are updating rapidly, therefore, our results could be interpreted with caution.
Author Contributions
C-WC and T-HT conceived the study. ZJ and W-YL collected the data. ZJ, W-YL, and Y-PY was responsible for the coding of the analyses. ZJ, W-YL, Y-PY, and T-HT analyzed and interpreted the data, and wrote the first draft of the paper. ZJ and W-YL searched, sorted, and interpreted the relevant literature. All authors contributed to the article and approved the submitted version.
Conflict of Interest
The authors declare that they do not have any conflicts of interest.
References
1. Sands, P, Mundaca-Shah, C, and Dzau, VJ. The Neglected Dimension of Global Security—A Framework for Countering Infectious-Disease Crises. New Engl J Med (2016) 374(13):1281–7. doi:10.1056/NEJMsr1600236
2.World Health Organization. WHO Coronavirus (COVID-19) Dashboard. Available from: https://covid19.who.int/ (Accessed January 13, 2022).
3. Fernandes, N. Economic Effects of Coronavirus Outbreak (COVID-19) on the World Economy (2020). doi:10.2139/ssrn.3557504
4. Haug, N, Geyrhofer, L, Londei, A, Dervic, E, Desvars-Larrive, A, Loreto, V, et al. Ranking the Effectiveness of Worldwide COVID-19 Government Interventions. Nat Hum Behav (2020) 4(12):1303–12. doi:10.1038/s41562-020-01009-0
5. Schaffer DeRoo, S, Pudalov, NJ, and Fu, LY. Planning for a COVID-19 Vaccination Program. JAMA (2020) 323(24):2458–9. doi:10.1001/jama.2020.8711
6. Khubchandani, J, Sharma, S, Price, JH, Wiblishauser, MJ, Sharma, M, and Webb, FJ. COVID-19 Vaccination Hesitancy in the United States: a Rapid National Assessment. J Community Health (2021) 46(2):270–7. doi:10.1007/s10900-020-00958-x
7. Wang, J, Jing, R, Lai, X, Zhang, H, Lyu, Y, Knoll, MD, et al. Acceptance of COVID-19 Vaccination During the COVID-19 Pandemic in China. Vaccines (2020) 8(3):482. doi:10.3390/vaccines8030482
8. Cascini, F, Pantovic, A, Al-Ajlouni, Y, Failla, G, and Ricciardi, W. Attitudes, Acceptance and Hesitancy Among the General Population Worldwide to Receive the COVID-19 Vaccines and Their Contributing Factors: A Systematic Review. EClinicalMedicine (2021) 40:101113. doi:10.1016/j.eclinm.2021.101113
9. Li, Y, Tenchov, R, Smoot, J, Liu, C, Watkins, S, and Zhou, Q. A Comprehensive Review of the Global Efforts on COVID-19 Vaccine Development. ACS Cent Sci (2021) 7(4):512–33. doi:10.1021/acscentsci.1c00120
10. Pormohammad, A, Zarei, M, Ghorbani, S, Mohammadi, M, Razizadeh, MH, Turner, DL, et al. Efficacy and Safety of COVID-19 Vaccines: A Systematic Review and Meta-Analysis of Randomized Clinical Trials. Vaccines (2021) 9(5):467. doi:10.3390/vaccines9050467
11. Zheng, C, Shao, W, Chen, X, Zhang, B, Wang, G, and Zhang, W. Real-world Effectiveness of COVID-19 Vaccines: A Literature Review and Meta-Analysis. Int J Infect Dis (2022) 114:252–60. doi:10.1016/j.ijid.2021.11.009
12. Cheng, H, Peng, Z, Luo, W, Mo, M, Zhou, H, Xin, X, et al. Efficacy and Safety of COVID-19 Vaccines in Phase III Trials: A Meta-Analysis. Vaccines (2021) 9(6):582. doi:10.3390/vaccines9060582
13. Chen, M, Yuan, Y, Zhou, Y, Deng, Z, Zhao, J, Feng, F, et al. Safety of SARS-CoV-2 Vaccines: A Systematic Review and Meta-Analysis of Randomized Controlled Trials. Infect Dis Poverty (2021) 10(1):94–12. doi:10.1186/s40249-021-00878-5
14. Greenhawt, M, Abrams, EM, Shaker, M, Chu, DK, Khan, D, Akin, C, et al. The Risk of Allergic Reaction to SARS-CoV-2 Vaccines and Recommended Evaluation and Management: A Systematic Review, Meta-Analysis, GRADE Assessment, and International Consensus Approach. J Allergy Clin Immunol Pract (2021) 9:3546–67. doi:10.1016/j.jaip.2021.06.006
15. Cai, C, Peng, Y, Shen, E, Huang, Q, Chen, Y, Liu, P, et al. A Comprehensive Analysis of the Efficacy and Safety of COVID-19 Vaccines. Mol Ther (2021) 29(9):2794–805. doi:10.1016/j.ymthe.2021.08.001
16. Papatheodorou, S. Umbrella Reviews: What They Are and Why We Need Them. Eur J Epidemiol (2019) 34:543–6. doi:10.1007/s10654-019-00505-6
17. Belbasis, L, Bellou, V, Evangelou, E, Ioannidis, JPA, and Tzoulaki, I. Environmental Risk Factors and Multiple Sclerosis: An Umbrella Review of Systematic Reviews and Meta-Analyses. Lancet Neurol (2015) 14(3):263–73. doi:10.1016/S1474-4422(14)70267-4
18. Higgins, JP, Thompson, SG, Deeks, JJ, and Altman, DG. Measuring Inconsistency in Meta-Analyses. BMJ (2003) 327(7414):557–60. doi:10.1136/bmj.327.7414.557
19. Solmi, M, Correll, C, Carvalho, A, and Ioannidis, JPA. The Role of Meta-Analyses and Umbrella Reviews in Assessing the Harms of Psychotropic Medications: Beyond Qualitative Synthesis. Epidemiol Psychiatr Sci (2018) 27(6):537–42. doi:10.1017/S204579601800032X
20. Fan, YJ, Chan, KH, and Hung, IF. Safety and Efficacy of COVID-19 Vaccines: A Systematic Review and Meta-Analysis of Different Vaccines at Phase 3. Vaccines (Basel) (2021) 9(9):989. doi:10.3390/vaccines9090989
21. Ling, Y, Zhong, J, and Luo, J. Safety and Effectiveness of SARS-CoV-2 Vaccines: A Systematic Review and Meta-Analysis. J Med Virol (2021) 93(12):6486–95. doi:10.1002/jmv.27203
22. McDonald, I, Murray, SM, Reynolds, CJ, Altmann, DM, and Boyton, RJ. Comparative Systematic Review and Meta-Analysis of Reactogenicity, Immunogenicity and Efficacy of Vaccines against SARS-CoV-2. NPJ Vaccin (2021) 6(1):74. doi:10.1038/s41541-021-00336-1
23. Liu, Q, Qin, C, Liu, M, and Liu, J. Effectiveness and Safety of SARS-CoV-2 Vaccine in Real-World Studies: A Systematic Review and Meta-Analysis. Infect Dis Poverty (2021) 10(1):132. doi:10.1186/s40249-021-00915-3
24. Wu, Q, Dudley, MZ, Chen, X, Bai, X, Dong, K, Zhuang, T, et al. Evaluation of the Safety Profile of COVID-19 Vaccines: A Rapid Review. BMC Med (2021) 19(1):173. doi:10.1186/s12916-021-02059-5
25. Alhumaid, S, Al Mutair, A, Al Alawi, Z, Rabaan, AA, Tirupathi, R, Alomari, MA, et al. Anaphylactic and Nonanaphylactic Reactions to SARS-CoV-2 Vaccines: A Systematic Review and Meta-Analysis. Allergy Asthma Clin Immunol (2021) 17(1):109. doi:10.1186/s13223-021-00613-7
26. Chen, J, Cai, Y, Chen, Y, Williams, AP, Gao, Y, and Zeng, J. Nervous and Muscular Adverse Events After COVID-19 Vaccination: A Systematic Review and Meta-Analysis of Clinical Trials. Vaccines (Basel) (2021) 9(8):939. doi:10.3390/vaccines9080939
27. Sharif, N, Alzahrani, KJ, Ahmed, SN, and Dey, SK. Efficacy, Immunogenicity and Safety of COVID-19 Vaccines: A Systematic Review and Meta-Analysis. Front Immunol (2021) 12:714170. doi:10.3389/fimmu.2021.714170
28. Uaprasert, N, Panrong, K, Rojnuckarin, P, and Chiasakul, T. Thromboembolic and Hemorrhagic Risks After Vaccination Against SARS-CoV-2: A Systematic Review and Meta-Analysis of Randomized Controlled Trials. Thromb J (2021) 19(1):86. doi:10.1186/s12959-021-00340-4
29. Sathian, B, Asim, M, Banerjee, I, Roy, B, Pizarro, AB, Mancha, MA, et al. Development and Implementation of a Potential Coronavirus Disease 2019 (COVID-19) Vaccine: A Systematic Review and Meta-Analysis of Vaccine Clinical Trials. Nepal J Epidemiol (2021) 11(1):959–82. doi:10.3126/nje.v11i1.36163
30. Naranbhai, V, Garcia-Beltran, WF, Chang, CC, Berrios Mairena, C, Thierauf, JC, Kirkpatrick, G, et al. Comparative Immunogenicity and Effectiveness of mRNA-1273, BNT162b2 and Ad26.COV2.S COVID-19 Vaccines. J Infect Dis (2021) 225:1141–50. doi:10.1093/infdis/jiab593
31. Kow, CS, and Hasan, SS. Real-world Effectiveness of BNT162b2 mRNA Vaccine: A Meta-Analysis of Large Observational Studies. Inflammopharmacology (2021) 29(4):1075–90. doi:10.1007/s10787-021-00839-2
32. Chandan, S, Khan, SR, Deliwala, S, Mohan, BP, Ramai, D, Chandan, OC, et al. Postvaccination SARS-CoV-2 Infection Among Healthcare Workers: A Systematic Review and Meta-Analysis. J Med Virol (2021) 94:1428–41. doi:10.1002/jmv.27457
33. Lurie, N, Saville, M, Hatchett, R, and Halton, J. Developing Covid-19 Vaccines at Pandemic Speed. N Engl J Med (2020) 382(21):1969–73. doi:10.1056/NEJMp2005630
34. Chen, Y, Klein, SL, Garibaldi, BT, Li, H, Wu, C, Osevala, NM, et al. Aging in COVID-19: Vulnerability, Immunity and Intervention. Ageing Res Rev (2021) 65:101205. doi:10.1016/j.arr.2020.101205
35. Sharma, O, Sultan, AA, Ding, H, and Triggle, CR. A Review of the Progress and Challenges of Developing a Vaccine for COVID-19. Front Immunol (2020) 11. doi:10.3389/fimmu.2020.585354
36. Duan, L, Zheng, Q, Zhang, H, Niu, Y, Lou, Y, and Wang, H. The SARS-CoV-2 Spike Glycoprotein Biosynthesis, Structure, Function, and Antigenicity: Implications for the Design of Spike-Based Vaccine Immunogens. Front Immunol (2020) 11. doi:10.3389/fimmu.2020.576622
37. Premkumar, L, Segovia-Chumbez, B, Jadi, R, Martinez, DR, Raut, R, Markmann, A, et al. The Receptor-Binding Domain of the Viral Spike Protein Is an Immunodominant and Highly Specific Target of Antibodies in SARS-CoV-2 Patients. Sci Immunol (2020) 5(48):eabc8413. doi:10.1126/sciimmunol.abc8413
38. Soleimanpour, S, and Yaghoubi, A. COVID-19 Vaccine: Where Are We Now and Where Should We Go? Expert Rev Vaccin (2021) 20(1):23–44. doi:10.1080/14760584.2021.1875824
39. Dong, Y, Dai, T, Wei, Y, Zhang, L, Zheng, M, and Zhou, F. A Systematic Review of SARS-CoV-2 Vaccine Candidates. Signal Transduction and Targeted Therapy. Signal Transduct Target Ther (2020) 5(1):1–237. doi:10.1038/s41392-020-00352-y
40. Creech, CB, Walker, SC, and Samuels, RJ. SARS-CoV-2 Vaccines. JAMA (2021) 325(13):1318–20. doi:10.1001/jama.2021.3199
41. Sadarangani, M, Marchant, A, and Kollmann, TR. Immunological Mechanisms of Vaccine-Induced Protection Against COVID-19 in Humans. Nat Rev Immunol (2021) 21(8):475–84. doi:10.1038/s41577-021-00578-z
42. Kumar, N, Sood, D, and Chandra, R. Design and Optimization of a Subunit Vaccine Targeting COVID-19 Molecular Shreds Using an Immunoinformatics Framework. Rsc Adv (2020) 10(59):35856–72. doi:10.1039/d0ra06849g
43. Harvey, WT, Carabelli, AM, Jackson, B, Gupta, RK, Thomson, EC, Harrison, EM, et al. SARS-CoV-2 Variants, Spike Mutations and Immune Escape. Nat Rev Microbiol (2021) 19(7):409–24. doi:10.1038/s41579-021-00573-0
44. Morris, KV. The Improbability of the Rapid Development of a Vaccine for SARS-CoV-2. Mol Ther (2020) 28(7):1548–9. doi:10.1016/j.ymthe.2020.06.005
45. Graham, BS. Rapid COVID-19 Vaccine Development. Science (2020) 368(6494):945–6. doi:10.1126/science.abb8923
46. Richmond, P, Hatchuel, L, Dong, M, Ma, B, Hu, B, Smolenov, I, et al. Safety and Immunogenicity of S-Trimer (SCB-2019), a Protein Subunit Vaccine Candidate for COVID-19 in Healthy Adults: A Phase 1, Randomised, Double-Blind, Placebo-Controlled Trial. The Lancet (2021) 397(10275):682–94. doi:10.1016/S0140-6736(21)00241-5
47. Polack, FP, Thomas, SJ, Kitchin, N, Absalon, J, Gurtman, A, Lockhart, S, et al. Safety and Efficacy of the BNT162b2 mRNA COVID-19 Vaccine. N Engl J Med (2020) 383:2603–15. doi:10.1056/NEJMoa2034577
48. Leung, T, Chan, A, Chan, E, Chan, VKY, Chui, CSL, Cowling, BJ, et al. Short-and Potential Long-Term Adverse Health Outcomes of COVID-19: A Rapid Review. Emerging Microbes Infect (2020) 9(1):2190–9. doi:10.1080/22221751.2020.1825914
Keywords: COVID-19, vaccination, effectiveness, safety, umbrella meta-analysis
Citation: Jiesisibieke ZL, Liu W-Y, Yang Y-P, Chien C-W and Tung T-H (2023) Effectiveness and Safety of COVID-19 Vaccinations: An Umbrella Meta-Analysis. Int J Public Health 68:1605526. doi: 10.3389/ijph.2023.1605526
Received: 25 October 2022; Accepted: 19 June 2023;
Published: 07 July 2023.
Edited by:
Olaf Von Dem Knesebeck, University Medical Center Hamburg-Eppendorf, GermanyReviewed by:
Antonio Vallinoto, Federal University of Pará, BrazilCopyright © 2023 Jiesisibieke, Liu, Yang, Chien and Tung. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Tao-Hsin Tung, dongdx@enzemed.com