News media reports about scientific failures that do not recognize the self-correcting nature of science can damage public perceptions of trust and confidence in scientific work, according to findings by researchers at the Annenberg Public Policy Center (APPC) of the University of Pennsylvania and the University at Buffalo, the State University of New York.
News stories about science follow several specific narratives, the researchers write in a new study in the journal Public Understanding of Science. One is that science is “in crisis” or “broken,” a narrative driven in recent years by reports of unsuccessful efforts to replicate findings in psychology, a rise in retractions, failures of peer review, and the misuse of statistics, among other things.
“Attempts and failures to replicate findings are an essential and healthy part of the scientific process,” said co-author Yotam Ophir, an assistant professor of communication at the University of Buffalo and a former postdoctoral fellow in APPC’s science of science communication program, where the work was conducted. “Our research shows the need for journalists and scientists to accurately contextualize such failures as part of the self-correcting nature of science.”
In an experiment, nearly 4,500 U.S. adults were assigned to read one of four different types of news stories about science or a control story. Among the findings:
- Exposure to stories highlighting problems reduced trust in scientists and induced negative beliefs about scientists.
- Greater effects were seen among people who read stories saying that science was in crisis or broken.
“We’ve identified a tendency in news coverage to overgeneralize the prevalence of problems in science and take them as an indicator that the enterprise as a whole is broken,” said co-author and APPC Director Kathleen Hall Jamieson. What the experiment found, she added, is that “exposure to news that mistakenly concluded that because something has gone wrong science is in crisis can unjustifiably undercut confidence in science.”
The experiment
The study sought to provide experimental evidence about the effects of exposure to different narratives about science. It was conducted online with 4,497 U.S. adults in early 2019 – before, Jamieson noted, the world was in the throes of the Covid-19 pandemic and “science discovered life-saving vaccines with unprecedented speed.”
The experiment tested the effects of four narratives:
- The “honorable quest” or discovery, in which a scientist discovers knowledge that is reliable and consequential;
- The “counterfeit quest,” or retraction of published work, in which a scientist engages in dishonorable and guileful conduct;
- The science is “in crisis/broken” narrative, which indicts scientists or the institution of science for failing to address a known problem; and
- The “problem explored,” where scientists explore and potentially fix a problem revealed by the “crisis/broken” narrative.
Participants were randomly assigned a reading based on edited news stories that were consistent with one of the narratives. For example, one “quest” story told of a discovery in immunotherapy to treat leukemia, while a “counterfeit quest” story described retracted scientific claims about eating behavior. A “science is broken” story described an “alarming increase in the number of retractions,” and a “problem explored” story looked at psychologists exploring ways to increase the reliability of psychology studies. A fifth group of participants read a control story about an unrelated subject, baseball.
After completing the readings, the participants were asked about their trust in science, beliefs about science, and support for funding of science.
Trust in science is high
The researchers found that:
- Trust in science was moderately high;
- Beliefs that science is self-correcting and beneficial were moderate to high;
- Among people with higher levels of trust in science, the more they perceived the problem-focused stories to be representative of science, the more likely they were to believe that science is self-correcting;
- For people with lower levels of trust in science, the effect was reversed: the more they saw the problem-focused stories as representative, the less likely they were to believe that science is self-correcting;
- Support for funding science was not affected by the stories.
“This study,” the authors concluded, “demonstrates the adverse, if small, effects of problem-focused media narratives on trust in, beliefs about, and support for scientists and points to the importance of perceived representativeness and audience trust in scientists in the audience’s response to them.”
The experiment follows up on a 2018 study by Jamieson in the Proceedings of the National Academy of Sciences. The earlier study examined three media narratives about science – the honorable quest, counterfeit quest, and crisis/broken. Of the “crisis/broken” articles examined in that study, just 29% indicated that science is self-correcting and 34% were written by a scientist. That study expressed concern that “defective narratives can enhance the capacity of partisans to discredit areas of science… containing findings that are ideologically uncongenial to them.” (Read more about the 2018 study.)
How journalists and scientists can bolster trust in science
“By labeling problems in scientific research ‘a crisis’ and by framing scientific failures as indications that science is unreliable, both scientists and journalists are failing to communicate the true values of science,” Ophir said. “Making mistakes is part of science. What the news media and scientists themselves often frame as failure is an indicator of healthy science.”
The content analysis found that honorable quest story was the most prevalent. But the study noted that when media reports do discuss failures “they tend to ignore scientific attempts to address the problems,” the authors write. “We argue that such narratives about individual or systemic scientific failures fail to communicate scientific norms of continuing exploration, scrutiny, and skepticism and could, particularly if being presented regularly and consistently, harm public trust and confidence in scientific work.”
Use of the “problem explored” narrative could lessen the detrimental effects and improve attitudes toward science by “better communicating scientific norms of continuing exploration, scrutiny, and skepticism,” the authors write. “As scientific communication in news media is the result of a negotiation between scientists and journalists, these results could guide future science communication efforts by both journalists and members of the scientific community.
“Like others before us…” they conclude, “we believe that such a change will require scientific institutions to reconsider the current incentive structure, that prioritizes the promotion of novel, statistically significant discoveries over [rigorous] self-correction efforts.”
The content analysis that grounded the study was supported by a grant from the Rita Allen Foundation.
“The effects of media narratives about failures and discoveries in science on beliefs about and support for science” was published in Public Understanding of Science. DOI: 10.1177/09636625211012630
News coverage of this study includes:
- Vaccine misinformation and social media (The Naked Scientist, June 14, 2021)
- Reporting on scientific failures and holding the scientific community accountable: 5 tips for journalists (Journalist’s Resource, June 8, 2021)
- By changing their framing of scientific failures and discoveries, journalists can bolster trust in science: New research (Journalist’s Resource, May 27, 2021)
