Why an analysis of COVID-19 vaccines from Floridas surgeon general is flawed – PolitiFact

Why an analysis of COVID-19 vaccines from Floridas surgeon general is flawed

In a Dec.16, 2020 file photo, a Pfizer BioNTech COVID-19 vaccine is administered at the Arizona Department of Health Services State Laboratory in Phoenix. (AP)

If youre a male Floridian between the ages of 18 and 39, recent news about the COVID-19 vaccines might have been alarming.

Florida Surgeon General Dr. Joseph Ladapo and the states department of health released new guidance recommending against the COVID-19 mRNA vaccines for that demographic. They said there was an 84% increase in cardiac-related deaths for vaccinated men in that age group.

"If it had been known two years ago or so that this vaccine would increase cardiac deaths in young men by 84%, would they have approved it? The obvious answer is, no," Ladapo said during an Oct.10 interview on "Tucker Carlson Tonight." "You would never give something to someone who is young and healthy and increase their risk of dying from sudden cardiac death by 84%."

But doctors and epidemiologists pushed back, saying the analysis that the recommendation is based on is flawed.

In the back-and-forth, what should you know about the analysis and recommendation?

We found that the data in the study is limited and does not paint a clear picture. The study wasnt peer-reviewed or published in a scientific journal, which means its accuracy wasnt measured by other scholars. And the new recommendation ignores important context about the vaccines safety record.

Dr. Daniel Salmon, a vaccinologist and director of the Institute for Vaccine Safety at Johns Hopkins University, said the studys conclusions may sound scary, but they don't mean much.

"Firstly, if they did a good study, they should write it up appropriately, with sufficient detail. And they didn't do that. So posting a report with insufficient detail, flawed data, that didn't consider the vaccine benefits and then taking this 84% and making a big deal out of it. And then saying that young men shouldnt get vaccinated, it's really irresponsible," Salmon said.

The data is problematic

The objective of the health departments analysis was "to evaluate the risks of all-cause and cardiac-related mortality following COVID-19 vaccination," it said.

"Far less attention has been paid to safety and the concerns of many individuals have been dismissed," Ladapo said when the analysis was released Oct. 7. "These are important findings that should be communicated to Floridians."

The study examined the death certificates of patients who died within a 25-week period after COVID-19 vaccination.

But, doctors pointed out, the number of deaths within the demographic in question was tiny.

"The results for males age 18-39 who received an mRNA vaccine included a grand total of 20 deaths during the risk period," Dr. Kristen Panthagani wrote in her medical blog, You Can Know Things. "If even just a few of these have another cause (which is highly likely), they lose statistical significance." Panthagani is an emergency medicine resident at Yale University who wrote extensively about the Florida analysis on Twitter and her blog.

Dr. Robert Wachter, chair of the department of medicine at the University of California at San Francisco, echoed that point in an interview with the Washington Post.

"Were talking about a very small number of deaths. An extra death or two would potentially change these results," Wachter said.

Another problem with the data is that the 25-week period used in the study was too long of a window, said Dr. Paul Offit, a member of the advisory committee to the U.S. Food and Drug Administration on the COVID-19 vaccines and the director of the Vaccine Education Center at Childrens Hospital of Philadelphia.

"Who does a study where you look at the effects 25 weeks later? I mean, if youre causing an immune response to your heart, that all happens within a week or 10 days," Offit said. "So you should know about that immediately, if theres a dramatic reaction to your heart."

We reached out to the Florida Department of Health to ask for any additional evidence they wanted us to consider. A spokesperson pointed us to Ladapos Twitter thread responding to critics of the analysis, which does not substantively address the issues raised.

Panthagani told PolitiFact that even the process of selecting the death records was questionable. The study selected records with specific ICD-10 codes, which she found concerning.

ICD-10 is short for International Classification of Diseases, 10th edition, which assigns numbers to disease diagnoses. When choosing which deaths to examine, state analysts selected only a handful of ICD-10 codes related to heart ailments, Panthagani said. ICD-10 codes can be a valid measure to use when examining deaths, but not always, Panthagani said. They dont fully explain the cause of death.

The Florida analysis included cardiac-related deaths "if their death record contained an ICD-10 code of I30-I52," it said. That covers conditions including endocarditis, valve disorders, myocarditis, cardiomyopathy, cardiac arrest, conduction system disorders, atrial fibrillation and flutter, heart failure, other cardiac arrhythmias, endocarditis and more, according to Science-Based Medicine.

"In short," Panthagani tweeted, "this list is too broad to be meaningful, excludes some cardiac issues but not others, and most of the diagnoses are far more likely to be caused by other ongoing disease processes rather than vaccination."

The study was neither peer reviewed nor published in a scientific journal

Typically, scientific studies undergo a rigorous process in order to prove their credibility, Offit said. That involves being submitted for peer review by scholars who look for flaws and confirm whether a study provides adequate proof of the theory being presented.

"You cant just stand up and declare something as being true without subjecting it to the normal process of having your data reviewed to see whether or not the data proves a point," Offit said.

Dr. David Gorski, a professor of surgery at Wayne State University School of Medicine and managing editor of Science-Based Medicine, had the same critique.

"The authors of the study are anonymous. Its funding source is not disclosed. Its methods section is opaque and makes it hard to figure out exactly how the analysis was done," Gorski said.

"I could easily publish a study on my blog and claim it shows something, and you would be right not to trust it or take it seriously. I might have taken this somewhat more seriously if the paper had disclosed the authors and stated that it was submitted to a decent peer-reviewed journal for publication. As it is, I just view this as propaganda."

We asked the Florida Department of Health why the study lacked authors names and was not peer-reviewed. We did not receive a response. But in an Oct. 11 interview with the Washington Post, Ladapo called it a "fake issue."

Analysis ignores that vaccine is safer than COVID-19

Spokespeople at the U.S. Department of Human Health and Services and the Food and Drug Administration pointed us to previously published guidance, which says it is overwhelmingly more safe to take the COVID-19 vaccine than to risk being unvaccinated and infected with COVID-19.

Floridas new recommendation "is flawed and a far cry from the science: COVID-19 vaccines have been proven safe and effective, and severe adverse reactions are rare," said FDA spokesperson Sarah Lovenheim in a statement. "The benefits of COVID-19 vaccination preventing death and hospitalization are well-established and continue to outweigh any potential risks."

Clinical trials were used to evaluate the vaccine in tens of thousands of participants, and since then, tens of millions have been vaccinated, Lovenheim said.

No vaccine is perfect. Rare cases of myocarditis, which is inflammation of the heart muscle, and pericarditis, inflammation in the outer lining of the heart, have been reported within several days after vaccination with mRNA vaccines.

A study published by the scientific journal JAMA Network reviewed data on the vaccine's safety from December 2020 to August 2021. The study found a small but increased risk of myocarditis for men after mRNA COVID-19 vaccines.

The Centers for Disease Control and Prevention wrote about the study and said that for 18-24 year olds males, there were 52.4 cases of myocarditis per million doses with Pfizer and 56.3 cases per million doses with Moderna. Symptoms of myocarditis are normally mild.

"Most patients with myocarditis or pericarditis after COVID-19 vaccination responded well to medicine and rest and felt better quickly," the CDC wrote.

Florida Department of Health, Exploring the relationship between all-cause and cardiac-related mortality following COVID-19 vaccination or infection in Florida residents: a self-controlled case series study, accessed Oct. 10, 2022

Florida Surgeon General Dr. Joseph Ladapo Twitter post, Oct. 7 2022

Tucker Carlson Tonight interview with Dr. Joseph Ladapo, Oct. 10, 2022

Florida Department of Health, State Surgeon General Dr. Joseph A. Ladapo Issues New Mrna COVID-19 Vaccine Guidance, Oct. 7, 2022

The BMJ, Self controlled case series methods: an alternative to standard epidemiological study designs, Sept. 12, 2016

Centers for Disease Control and Prevention, International Classification of Diseases, (ICD-10-CM/PCS) Transition - Background, accessed Oct. 11, 2022

Institute for Vaccine Safety, accessed Oct. 11, 2022

PolitiFact, Social media claim misleads on risk of myocarditis, Jan. 31, 2022

Science-Based Medicine, The State of Florida spreads antivaccine disinformation disguised as an epidemiological "study", Oct. 10, 2022

Dr. Kristen Panthagani Twitter thread, Oct. 8, 2022

Dr. Kristen Panthagani tweet, Oct. 8, 2022

You Can Know Things, A critical review of Floridas new vaccine analysis, Oct. 9, 2022

JAMA Network, Myocarditis Cases Reported After mRNA-Based COVID-19 Vaccination in the US From December 2020 to August 2021, Jan. 25, 2022

Centers for Disease Control and Prevention, Selected Adverse Events Reported after COVID-19 Vaccination, updated Oct. 12, 2022

The Washington Post, Experts slam Florida surgeon generals warning on coronavirus vaccines, Oct. 11, 2022

Florida Surgeon General Dr. Joseph Ladapo Twitter post, Oct. 7 2022

PolitiFact, Social media claim misleads on risk of myocarditis, Jan. 31, 2022

Interview with Dr. Daniel Salmon, director of the Institute for Vaccine Safety at Johns Hopkins University Oct. 10, 11

Phone interview with Dr. Paul Offit, director of the Vaccine Education Center and pediatrician in the div. Of infectious diseases at Childrens Hospital of Philadelphia Oct. 12, 2022

Phone Interview with Dr. Kristen Panthangani, resident at Yale Emergency Medicine and founder of You Can Know Things, Oct. 11

Email interview with Dr. David Gorski, professor of surgery and oncology at Wayne State University and man. Editor of Science Based Medicine.org, Oct. 10, 11

Email interview with Sarah Lovenheim, Food and Drug Administration spokesperson, Oct. 11, 2022

More:
Why an analysis of COVID-19 vaccines from Floridas surgeon general is flawed - PolitiFact