Your goofy but lovable cousin just told you that you should stop eating eggs because he read somewhere that a study showed they are bad for you.
How much should you trust your relative on such matters? More importantly, how much should you rely on one newly published bit of research when deciding what to make for breakfast?
To be clear, this is not an article about the health-promoting or health-torpedoing properties of eggs. It’s about how scientific knowledge is built piece by piece from many studies. What scientists know is refined over time as new results either do or don’t point to the same conclusion.
I’m a geographer who’s been doing and teaching science for many decades, with a sideline of teaching and writing about how science is done. Many people, quite understandably, take a single experiment or study as the be-all and end-all of knowledge because that’s how research often is presented by the press or on social media. But the better way to approach new research is to find how it weaves together with other work on the topic to create big-picture understanding.
How science works
Most research studies are undertaken either to fill a gap in our knowledge or to test an existing theory to see whether it deserves the confidence people have in it. After identifying the topic, scientists design a study to achieve those ends. They may run an experiment to learn more about how a chemical affects certain cells, for instance, or collect data in the field to track a natural phenomenon, such as how water temperatures affect hurricanes.
Then the researchers submit their findings to a peer-reviewed journal, where other experts – the scientists’ peers – decide whether it’s quality research deserving of publication.
Not all journals have rigorous peer review. Papers are highly unreliable if published by “paper mills” – journals that appear scholarly but will publish anything if the authors pay a fee.
Peer review doesn’t guarantee that the conclusions are valid, but it increases the chances that they are. Individual papers might be wrong because of honest mistakes, such as unforeseen limitations in the experimental design or, rarely, from outright fraud.
No scientific paper solves a problem once and for all. Neither does it negate all previous research. Well-done research contributes a bit to the scientific community’s understanding of a topic. The next, and crucial, step is putting individual studies in context with other research on the topic.
Even if there is current consensus, a new study may reveal a weakness, and that could lead to more research to figure out what is more likely to be correct. Scientific knowledge is constantly being refined as new information comes to light.
Adding more evidence bit by bit
One question to ask as you consider a particular finding is whether it has been directly replicated, meaning other researchers repeated the experiment to see whether they got the same results. Unfortunately, replication is relatively rare in science; more common are similar studies using comparable data, different methods, or both.
Your confidence can grow when scientists have performed a bunch of related research that’s gone through peer review, been published in scholarly journals and mostly points in the same direction. Of course, if they don’t agree, then your confidence should be weaker.
Sometimes researchers may compile these comparisons in what’s called a systematic review. They may use statistical techniques to perform meta-analysis on data from many different studies at once. Generally speaking, the more good data used to test an idea, the better.
An additional issue is how many studies have been done on a topic. There are thousands of studies on the causes of lung cancer, but there may be only one or two on how a couple of particular genes affect hair loss. Scientists’ confidence in what is known about lung cancer, then, is far greater than what is known about how those genes may have contributed to my baldness.
Appreciating the strength of the evidence is as important as understanding the evidence itself.
Get a helping hand
The idea of expertise has fallen out of favor in some quarters. But experts are vital when it comes to understanding scientific issues. An expert in this sense is someone who has been immersed in the topic for years, knows how to evaluate the relevant studies, and, ideally, has done research on it.
With such a background, an expert is a good judge of how likely any one study is to be wrong. Equally important, they also must try to control the all-too-human impulse to accept what they like and reject what they don’t.
Unfortunately, most people rarely have direct access to experts. The next best thing is someone educated in the general topic – verifiably educated, not someone who browses the internet for a few hours.
Healthcare professionals who have years of training, clinical experience and requirements to keep up with the literature in their field can help you make good decisions based on new medical research. But be careful. You want to rely on someone who updates their recommendations as the state of scientific knowledge evolves, but not someone who latches onto every new outlandish discovery.
In practice, some healthcare practitioners – hopefully a small minority – are not trustworthy on such matters. If someone is selling you something that sounds too good to be true, assume that it is. They may even have a financial or personal stake in their recommendation.
Consider the source
You should retain some skepticism about what you read in the popular press and even more about what you see on social media.
A good journalist who knows how to assess new studies can act as a guide and help you understand scientific issues. You’re looking for journalists who can accurately and objectively report on new research and help put it in context with what else is known. Unfortunately, there is no list of good versus bad journalists, but general guidance is available, such as that from nonprofit journalism organization The Trust Project.
Journalists who are well versed in how science works can also help you spot whether there are any conflicts of interest at play. Was that study that encourages staying energetic by eating a pound of candy a day sponsored by a snack food company? That would be a major red flag.
I’m not saying that everyone needs to do a thorough literature review before speaking about a scientific issue or deciding whether to eat eggs a couple of times a week. But I do encourage you to adopt a little humility about what you know and understand, along with a realistic appreciation for the limits of both your own knowledge and what the scientific community understands.
And definitely don’t make life-altering decisions based on an article describing one scientific study, even if your cousin tells you to.
Jeffrey A. Lee does not work for, consult, own shares in or receive funding from any company or organization that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.
This article was originally published on The Conversation. Read the original article.
