Three doses of COVID-19 mRNA vaccination are safe based on adverse events reported in electronic health records

1. Age greater than or equal to 18 years at the date of initial COVID-19 vaccination.
2. Received the first two doses of BNT162b2 or mRNA-1273 per-emergency use
authorization protocol. Per-protocol BNT162b2 vaccination is defined as two BNT162b2
doses administered 18-28 days apart. Per-protocol mRNA-1273 vaccination is defined as
two mRNA-1273 doses administered 25-35 days apart.
3. Received a 3rd dose of an mRNA-based COVID-19 vaccine at least 28 days after the 2nd
dose.
4. 3rd COVID-19 vaccine dose of the same type as the original 2 doses.
5. Did not have more than three doses of an mRNA-based COVID-19 vaccine on record.
6. Did not previously receive any doses of a non-mRNA based COVID-19 vaccine (e.g.
Janssen – Ad26.COV2.S).
7. At least 14 days of follow up after their 3rd vaccine dose.

Study participants are divided into four cohorts for analysis, depending on the vaccine types of
their initial two vaccine doses and their 3rd vaccine dose (Figure 1a). Specifically, cohorts of
individuals with three BNT162b2 vaccine doses (n = 38,094) and three mRNA-1273 vaccine
doses (n = 9,905). Demographic and clinical characteristics of the cohorts are shown in Table
S1, and information on the timing of vaccine doses is shown in Figure S1.

Extracting adverse event sentiments from EHR data using augmented curation
A BERT-based27 classification model was used to determine the sentiment of adverse
event phenotypes mentioned in the clinical notes. This model has been previously used to identify
signs and symptoms of COVID-19,28 short and long-term complications of COVID-19,29 and
adverse events of mRNA-based COVID-19 vaccines.23 Given a sentence which includes a
phenotype, this model outputs one of the following labels: Yes – confirmed diagnosis, Maybe –
possible diagnosis, No – ruled out diagnosis, or Other – none of the above (e.g. family history of
diagnosis). This model was trained on a dataset of 18,490 sentences from clinical notes in the
Mayo Clinic including over 250 different phenotypes and achieves an out-of-sample accuracy of
93.6% and precision and recall values above 95%.28
For this analysis, the above classification model was applied to clinical notes of each of the
individuals in the study population for the following time periods: -15 to -1 days prior to the 1st
vaccine dose, 0 to 14 days following the 1st vaccine dose, 0 to 14 days following the 2nd vaccine
dose, and 0 to 14 days following the 3rd vaccine dose (Figure 1b). 19 adverse event phenotypes
were considered, including: anaphylaxis, arthralgia, cerebral venous sinus thrombosis (CVST),
chills, diarrhea, erythema, facial paralysis, fatigue, fever, headache, local pain, local swelling,
lymphadenopathy, myalgia, myocarditis, nausea, pericarditis, soreness, and vomiting. For each
(adverse event, time period) pair, individuals with at least one clinical note labelled “Yes” by the
model with over 90% confidence were counted as having the adverse event. For a select set of
rare severe adverse events (anaphylaxis, facial paralysis, myocarditis, pericarditis), additional
manual curation was performed (by JCO and DWC) to confirm that the patients identified by the
model did experience the adverse events during the time period of interest and that the adverse
event was not attributed to another known cause (i.e., anaphylaxis due to allergic reaction to a
known non-vaccine allergen).

All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.
The copyright holder for this preprintthis version posted November 9, 2021. ; https://doi.org/10.1101/2021.11.05.21265961doi: medRxiv preprint Estimation of adverse event risk from EHR data
We used previously described augmented curation models to extract sentiments of adverse
events from clinical notes.23,28 Specifically, for each individual in a cohort, we determine whether
positive sentiments for vaccine-associated adverse events are present in their clinical notes,
during a specific 14-day period relative to their date of vaccination (Figure 1). The risk of an
adverse event is then reported as the percentage of vaccine recipients in a cohort with a positive
sentiment for that adverse event. This risk is compared to the baseline risk in the cohort, taken
as the risk for the adverse event in a 14-day period before the 1st COVID-19 vaccine dose. The
reported confidence intervals and p-values are determined using bootstrap resampling (N=10,000
samples).
Extracting comorbidity and immunosuppressant medication data from EHRs
For each patient, Elixhauser comorbidities30 for HIV/AIDS and cancer were determined
using ICD-10 codes in the 5 year period leading up to the 1st COVID-19 vaccine dose. In addition,
overall Elixhauser comorbidity scores were computed using the Van Walraven method.31 Patients
who had taken immunosuppressant medications in the past 1 year in their medical history were
identified by querying the EHR database for the list of medications in the drug class WHO ATC
LO4A.32

IRB approval for human subjects research
This study was reviewed and approved by the Mayo Clinic Institutional Review Board (IRB 20 –
003278) as a minimal risk study. Subjects were excluded if they did not have a research
authorization on file. The approved IRB was titled: Study of COVID-19 patient characteristics with
augmented curation of Electronic Health Records (EHR) to inform strategic and operational
decisions with the Mayo Clinic. The following resource provides further information on the Mayo
Clinic Institutional Review Board and adherence to basic ethical principles underlying the conduct
of research, and ensuring that the rights and well-being of potential research subjects are
adequately protected: http://www.mayo.edu/research/institutional-review-board/overview.

Results
Overall, we find no significant difference in the reporting of severe adverse events and a
significant increase in reporting for most low-severity adverse events after the 3rd COVID-19
vaccine dose, compared with earlier doses.
The most common adverse events reported after the 3rd vaccine dose were fatigue (4.92%),
lymphadenopathy (2.89%), nausea (2.62%), headache (2.47%), arthralgia (2.12%), myalgia
(1.99%), diarrhea (1.70%), erythema (1.00%), fever (1.11%), vomiting (1.10%), chills (0.47%),
and soreness (0.36%) (Figure S2). The median time between the vaccine dose and reporting of
the adverse events is listed in Table S2 and Table S3. We quantified increased reporting of
adverse events after the 3rd dose, compared with the 2nd dose and baseline incidence, using the
risk difference (RD). We found that compared to the 2nd dose, there was increased reporting of
most common adverse events for both BNT162b2 and mRNA-1273 after dose #3. Notably, overall
patients reported significantly more fatigue, RD=1.45% (1.20%-1.71% 95% CI),
lymphadenopathy, RD=0.82% (0.62%-1.01% 95% CI), nausea, RD=0.58% (0.39%-0.78% 95%
All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.
The copyright holder for this preprintthis version posted November 9, 2021. ; https://doi.org/10.1101/2021.11.05.21265961doi: medRxiv preprint CI), headache, RD=0.40% (0.21%-0.59% 95% CI), arthralgia, RD=0.42% (0.25%-0.60% 95% CI),
myalgia, RD=0.36% (0.19%-0.53% 95% CI), diarrhea, RD=0.46% (0.31%-0.62% 95% CI), fever,
RD=0.30% (0.17%-0.42% 95% CI), vomiting, RD=0.30% (0.18%-0.42% 95% CI), and chills,
RD=0.10% (0.02%-0.18% 95% CI) (Figure 2). Risk differences are also reported per vaccine
brand (Figure 2a-b). Notably, no vaccine specific adverse events were found.
Reporting of severe adverse events was rare after the 3rd dose and was not significantly increased
compared with the frequency of those events after the 2nd dose (Figure S2). After the 3rd vaccine
dose, 4 patients reported pericarditis (0.01%, 0%-0.01% 95% CI), 2 patients reported anaphylaxis
(0.00%, 0%-0.01% 95% CI), and 1 patient reported myocarditis (<0.01%, 95% CI)). We find no
significant increase in risk (p-value < 0.05) of these adverse events after the 3rd dose, compared
with after the 2nd dose (Figure S2). Additionally, we assessed the rate of emergency department
visits within 2-days of each vaccine dose and found only a slight increase after the 3rd BNT162b2
dose, 0.29% of vaccine recipients (0.24%-0.35%, 95% CI) compared with after the 2nd dose, 0.2%
of vaccine recipients (0.15%-0.24%, 95% CI) (Figure S3). No significant difference in emergency
department visits was found for mRNA-1273 recipients.

Discussion
Our results suggest that a 3rd dose of the same type of vaccination after either a BNT162b2
or mRNA-1273 primary series is safe. Although we observed an increase in early post-vaccination
adverse events after the 3rd dose compared to earlier doses, these were for symptoms of low
concern (i.e., fatigue, lymphadenopathy, nausea, and diarrhea). We observed no significant
increase in EHR reporting of severe adverse events after the 3rd dose compared with after the 2nd
dose, with incidence comparable with previous literature.16,17 The observed increase in adverse
events compared with earlier doses could be due to a stronger response elicited by the 3rd dose,
comparable to what was observed for the 2nd dose compared with the 1st dose.1,2,33 Further studies
are needed to explore whether the 3rd vaccine dose does indeed induce a stronger immune
response.
This study has some limitations. To control for patient-specific covariates that may impact
symptoms experienced after vaccination, we have only considered cohorts of patients that
received exactly three doses of mRNA-based COVID-19 vaccines. No comparison between the
adverse events reported in the BNT162b2 and mRNA-1273 cohorts should be made, as these
cohorts differ in potentially confounding factors, including in the relative rate of
immunosuppressed individuals (Table S1). The cohorts are also less likely to include individuals
that had strong adverse reactions to earlier doses of mRNA-based COVID-19 vaccines, as such
individuals are more likely to opt out of additional vaccination.15 Indeed, analysis of individuals
that received exactly one dose of an mRNA-based COVID-19 vaccine showed a significantly
greater RD of adverse events after dose one, compared with baseline, than what is observed in
the 3-dose cohort (Figure S4). Additionally, a large proportion in the 3-dose cohort is
immunosuppressed and of advanced age (Table 1), potentially reducing their immune reaction to
vaccination and associated adverse events; and resulting in lower prevalence of adverse events
that are more prevalent in younger individuals (i.e., myocarditis). This means that our conclusions
on the safety of additional dose vaccination with BNT162b2 or mRNA-1273 apply specifically to
individuals that are included in this cohort, opted to receive additional COVID-19 mRNA vaccine
dose of the same type and opted to report symptoms to allow EHR capture. Therefore, these
All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.
The copyright holder for this preprintthis version posted November 9, 2021. ; https://doi.org/10.1101/2021.11.05.21265961doi: medRxiv preprint results may not be generalizable to otherwise healthy individuals. Further, we have no way of
accounting for any variation in the likelihood of individuals to report outcomes; e.g. if individuals
were more inclined to report certain effects after a 1st vaccine dose, but are more likely to dismiss
them after a 3rd dose without contacting their provider, we would not be able to detect that with
our data. Our eligible study population included too few individuals with mixed vaccine brands (n
= 887) or with 2nd doses of Ad26.COV2.S (n = 76) to reach meaningful conclusions, and these
populations were therefore not included in the present report. Further study on larger more
general populations might therefore find increased incidence of adverse events and will be
needed to reach meaningful conclusions on the prevalence of rare adverse events.
Here, we have quantified the clinical symptoms experienced by vaccine recipients by
augmented curation of EHR notes. The augmented curation process involves defining a list of
symptoms and subsequently quantifying positive sentiments for these symptoms in EHR notes.
As symptoms that were not explicitly included will not be quantified, the study design is not
suitable for discovering adverse events that have not previously been associated with COVID-19
vaccination. Identification of positive sentiments for vaccine adverse events using augmented
curation is not perfect, however previous studies have demonstrated excellent accuracy of the
used augmented curation algorithms for related tasks.28
Extraction of adverse events from EHR notes is complementary with the clinical trials and
self-reporting approaches used in previous studies. Barriers associated with self-reporting of
adverse events (i.e., via a survey or device) are removed, and all adverse events for which an
individual seeks medical attention will be counted. This reduction of barriers to data collection
allowed us to analyze a larger cohort of 3-dose vaccine recipients than in previous studies, without
necessary selection for individuals willing to self-report. However, we will only detect symptoms
that individuals reported to clinicians and that were captured in EHR notes. This likely results in
the low rate of common but non-severe adverse events (i.e., fatigue, local swelling, and local
redness) compared with previous studies,24,26 as individuals may not seek medical attention for
expected low-severity adverse events after vaccination.
This study provides further evidence that 3rd dose vaccination with the same type of
COVID-19 mRNA vaccine as used in the primary series is safe in high-risk populations. Together
with previous studies of booster dose safety13,25,26 and effectiveness,12,13,24 our study supports 3rd
dose mRNA COVID-19 vaccination of at-risk populations.

Data Availability

After publication, the data will be made available upon reasonable requests to the corresponding
author. A proposal with a detailed description of study objectives and the statistical analysis plan
will be needed for evaluation of the reasonability of requests. Deidentified data will be provided
after approval from the corresponding author and the Mayo Clinic.
Competing Interests
MJN, CP, ES, GD, PJL, AJV, and VS are employees of nference and have financial interests i n
the company. nference is collaborating with bio-pharmaceutical companies on data science
initiatives unrelated to this study. These collaborations had no role in study design, data collection
All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.
The copyright holder for this preprintthis version posted November 9, 2021. ; https://doi.org/10.1101/2021.11.05.21265961doi: medRxiv preprint and analysis, decision to publish, or preparation of the manuscript. JCO has received small grants
from nference, Inc., and personal consulting fees from Bates College and Elsevier Inc. All of these
activities are outside of the present work. MDS receives research funding for the HEROES
Together vaccine SE registry from Pfizer via Duke University. AV reports being an inventor for
Mayo Clinic Travel App interaction with Smart Medical Kit and Medical Kit for Pilgrims. ADB is
supported by grants from NIAID (grants AI110173 and AI120698) Amfar (#109593) and Mayo
Clinic (HH Shieck Khalifa Bib Zayed Al-Nahyan Named Professorship of Infectious Diseases).
ADB is a paid consultant for Abbvie, Gilead, Freedom Tunnel, Pinetree therapeutics Primmune,
Immunome, MarPam, and Flambeau Diagnostics, is a paid member of the DSMB for Corvus
Pharmaceuticals, Equilium, and Excision Biotherapeutics, has received fees for speaking for
Reach MD and Medscape, owns equity for scientific advisory work in Zentalis and nference, and
is founder and President of Splissen therapeutics. DWC, JH, JEG, HLG, and LLS have no
interests to disclose. This research has been reviewed by the Mayo Clinic Conflict of Interest
Review Board and is being conducted in compliance with Mayo Clinic Conflict of Interest policies.

All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.
The copyright holder for this preprintthis version posted November 9, 2021. ; https://doi.org/10.1101/2021.11.05.21265961doi: medRxiv preprint
Figures


Figure 1: Schematic depiction of the study design. (a) The study population consists of 48,799
adult patients at Mayo Clinic Health System that meet all study inclusion criteria. This study
population is subdivided into cohorts depending on the vaccine doses they received. ( b) To find
significant adverse events associated with COVID-19 vaccination, we quantified clinical
symptoms reported in EHR notes during 14-day periods starting at the date of each vaccine dose
(orange), as well as a control 14-day period before the 1st vaccine dose (blue).

All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.
The copyright holder for this preprintthis version posted November 9, 2021. ; https://doi.org/10.1101/2021.11.05.21265961doi: medRxiv preprint
Figure 2: Risk difference for common adverse events after the 3rd vaccine dose compared with
after the 2nd dose vaccine dose (orange) and baseline risk (blue). The risk difference is shown
for 3-dose BNT162b2 recipients (a) and for 3-dose mRNA-1273 recipients (b). Error bars indicate
95% confidence intervals, and asterisks indicate adverse events with significant (two -tailed p-
value < 0.05) difference in prevalence after the 3rd dose, compared with before the 1st dose (blue
asterisk) or after the 2nd dose (orange asterisk).


All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.
The copyright holder for this preprintthis version posted November 9, 2021. ; https://doi.org/10.1101/2021.11.05.21265961doi: medRxiv preprint
Figure S1: Distribution of the time between vaccine doses, for individuals that received three
doses of BNT162b2 (a) and individuals that received three doses of mRNA-1273 (b)

Votre commentaire

Entrez vos coordonnées ci-dessous ou cliquez sur une icône pour vous connecter:

Logo WordPress.com

Vous commentez à l’aide de votre compte WordPress.com. Déconnexion /  Changer )

Image Twitter

Vous commentez à l’aide de votre compte Twitter. Déconnexion /  Changer )

Photo Facebook

Vous commentez à l’aide de votre compte Facebook. Déconnexion /  Changer )

Connexion à %s