Framing early COVID-19 research as uncertain: A mixed-method study of public reactions. Summary During the COVID-19 pandemic, journalists have been encouraged to convey uncertainty around preliminary scientific evidence, including mentioning when research is not published or has not been verified by peer review. To understand how public audiences interpret this information, we conducted a mixed-method study with American adults. Participants read a news article about preliminary COVID-19 vaccine research in early April 2021, just as the vaccine was widely available to the American public. We modified the article to test two ways of conveying uncertainty (coverage of scientific claims and mention of preprint status) in a 2 × 2 between-participants factorial design. To complement this, we collected open-ended data to assess participants’ understanding of the concept of a scientific preprint. In total, participants who read covered (vs. uncovered) versions of the article reported less favorable vaccine attitudes and intentions and found scientists and news reports to be less trustworthy. These effects were moderated by participants’ epistemic beliefs and their preference for information about scientific uncertainty. However, there was no impact on the study’s description as a preprint , and participants’ qualitative responses indicated limited understanding of the concept . We discuss the implications of these findings for communicating initial scientific evidence to the public and outline important next steps for research and theory building. |
Medical epistemic beliefs
Epistemic beliefs reflect people’s views about the nature of knowledge and the process of knowledge creation. In the context of medicine, people may perceive scientific knowledge as evolutionary and fallible (i.e., viewing it as a continuous process) or as stable and unchanging (i.e., viewing it as fixed). In a recent study, people who considered science to be stable and unchanging were more likely to say they expected journalists to provide definitive information about the COVID-19 pandemic. It stands to reason that beliefs about the nature of medical science would affect a person’s response to disclosure of scientific uncertainty about COVID-19 vaccines. Those who see science as a process probably expect uncertainty about new discoveries, and for them disclosure is a gesture of transparency, while claims of certainty can raise suspicion. Furthermore, believing that uncertainty is inherent to science might make one more apt to recognize its revelation. In contrast, those who believe that scientific knowledge is fixed may view scientific uncertainty as a marker of low-quality research and (to the extent they perceive it) react unfavorably to its dissemination.
Comments
People don’t know what a preprint is. Here’s why that matters
New research from the University of Georgia suggests that most people don’t understand the difference between a preprint and an article published in an academic journal.
Preprints are research papers that have not undergone peer review, the process by which study findings are validated by experts who were not involved in the research . The study found that most readers have little or no understanding of what a preprint actually is. That lack of understanding could lead to public distrust in science, as findings and how those findings are described can change between the preprint phase and publication after peer review. Frequent reporting on scientific preprints could also damage trust in the news.
Preprints used to circulate primarily within scientific communities, but the COVID-19 pandemic led to an unprecedented number of preprints flooding the internet. The desire to obtain information as quickly as possible was understandable, the researchers said. But it also sets a problematic precedent.
"With preprints, there are still uncertainties that haven’t been resolved," said Chelsea Ratcliff, lead author of the study and assistant professor in the department of communication studies in the Franklin College of Arts and Sciences. “Many preprints are not even published. I really think it’s important for the public to understand that.
"If people base their attitudes, for example, about a new drug on the evidence in a preprint or if they base their health decisions on a preprint , they should be able to get a sense of its preliminary nature ."
75% of people don’t know what a prepress is
The researchers assigned 415 people to read news articles about preliminary COVID-19 research. The preprint focused on the relationship between COVID-19 vaccine side effects and vaccine effectiveness. One group read stories in which the study was described as “a preliminary study recently published online and not yet evaluated by outside experts,” while the other version simply described it as a “study.”
The news article also presented the research conclusions in a tentative way, such as saying that the findings "suggest" or "could" mean that people are protected against COVID-19, regardless of whether they experienced side effects from the vaccines, or They portrayed them as if they were true.
As expected, participants rated the findings as less certain when the story said the findings were tentative. But using the word "preprint" in the text and mentioning that the research had not undergone peer review had no effect on participants’ interpretation of the study. Both groups rated the certainty of the research the same.
When researchers asked participants to describe what they think "preprint" means when it appears in a science news story, 75% gave a definition that showed they didn’t really understand the concept .
“What I tell my students is to think of any study as a drop in the bucket of knowledge about a phenomenon,” Ratcliff said. “ No single study proves or disproves anything , and we may need an extra degree of caution when it comes to a preprint study.” “I see value in preprints, but simply telling the public that it is a preprint is not enough to give them the feeling that it is preliminary evidence.”
"We need to find other strategies to communicate about preprints effectively," added Alice Fleerackers, co-author of the study and doctoral candidate at Simon Fraser University. "Simply labeling the research as a ’preprint’, even with a brief definition, does not seem to move the needle."
Journalists reporting on preprint studies should briefly explain the academic peer review process and warn readers that preprint findings are subject to change, the researchers said .
“Before the pandemic, which is when reporting on preprints really skyrocketed, the main purpose of a preprint was for scientists to share their findings with other scientists,” Ratcliff said. “They were not intended to influence public policies, attitudes or behaviors. And that’s something worth keeping in mind for readers.”
Conclusion
Testing public responses to explicit mention of preprint status in media coverage of scientific research is a novel contribution of this study, with relevance to the communication of COVID-19 evidence and to scientific and health journalism in general. . We examine the effects of two forms of preprint science uncertainty disclosure on COVID-19 vaccines. In line with other research, the transmission of scientific uncertainty through coverage produced lower ratings of credibility of news articles and credibility of scientific sources, and lower intention to receive a COVID-19 vaccine, among certain segments of the sample. However, revealing the preprint status had no effect. To complement and contextualize these results, we used an open-ended question to assess participants’ understanding of the concept of scientific preprint . In general, participants exhibited a low understanding of the concept . Given the likelihood that preprints will continue to be a tool that scientists use to disseminate their research, additional work examining the implications of preprints on the public’s understanding of research, science, and publishing will continue to be a pressing topic for health communication scholarship.
Published in Health Communication , the study was co-authored by Rebekah Wicke, Blue Harvill, Andy King and Jakob Jensen.
