Medical news: A case for skepticism

When you read medical news, do you ever get drawn in by the headline only to find the details deliver something quite different (or less) than expected? Or do the findings sound so dramatic that you wonder whether the results might be exaggerated or misleading? If you answered yes, I'm with you.

The reasons to be skeptical are many. And it's not that there are evil people out there deliberately trying to mislead you — well, there are a few of those, but only a few. Pressures on those that bring us health news make it almost certain that at times, information will be biased, incomplete, or flat-out wrong.

I'm not talking about research findings that seem misleading because they are eventually debunked by future studies. For example, only a few years ago, doctors believed hormone replacement therapy was good for the cardiovascular health of postmenopausal women. Better research eventually proved just the opposite. What I am talking about is current research as published in reputable medical journals and reported on your nightly news, in your newspaper, or online. Even if the research findings are proven to be true, the "spin" can be a bit (or a lot) over the top.

The case for skepticism

The case for skepticism can be made at just about every step of the way that medical news makes its way to us. For example:

• The design of the clinical trial. Studies with positive findings are more likely to get published than those with negative (and often disappointing) results. And research is often funded by pharmaceutical companies that have little incentive to perform high-quality, expensive studies if the results might hurt chances for FDA approval or future sales. That's why many trials are designed in a way that enhances the chances of success for the new drug; for example, rather than comparing a new drug against a competitor's drug, it might be compared to a placebo or an old drug at a dose too low to be effective.
• Authors. Academic promotion, better jobs, higher pay, tenure, and prestige all ride on the ability to publish. So, even if the research is preliminary, applies to few people, or is of uncertain impact, casting the research in the most favorable and dramatic light is standard operating procedure. Sometimes, this becomes clear when the study makes sweeping generalizations about results that only apply to a few.
• Editors. As mentioned, there is a general tendency for journals to favor positive results over negative findings, as they garner more readership — and journals want readers. Journal editors also may not have the expertise to critically assess a study. All of this conspires to encourage publishing research that is likely to have positive findings, especially if couched in dramatic terms.
• Press releases. A company press release can have far-reaching impact, including on its stock price. There are regulations that place legal liability for a company overstating or misstating research results. So, press releases are carefully scrutinized by company lawyers. Even so, there are many examples of overstatement, selective disclosure of results, and understating limitations. In 2013, a biotech CEO was sentenced to six months of house arrest for overstating the results of his company's drug. A more recent example is a press release that touted a company's "breakthrough" drug for a type of brain tumor and noted that the drug was already approved by the FDA. In fact, it had been approved for dogs (not humans) and had never been studied in people with the type of brain tumor mentioned in the press release.
• Medical reporters. Some medical reports simply summarize or repeat what a press release says, often not explaining the limitations of the study or overstating the importance of the results. With pressure to grab a reader's or viewer's attention, there is a tendency to err on the side of overstatement. I've frequently seen headlines that caught my attention only to be disappointed at how poorly that headline described the study. A recent example: "Do Away with BMI, Study Says." But that's not what the study said. The study suggested that when it came to cardiovascular health (as defined by important measures such as cholesterol levels, blood sugar, and blood pressure), body mass index (BMI) alone was not terribly accurate. That doesn't mean that BMI has no value in combination with other measures or for other conditions – and it doesn't mean we should "throw it out."

How to read health news with a critical eye

Be skeptical. There's a reason that most good medical news stories end with an interview of someone not involved in the research: to provide an unbiased bit of balance and perspective on the story. What that expert usually says is something like: "these findings are interesting but we need more research to know how important it is." The fact is, advances in medicine are incremental and usually slow. It's worth noting that the Nobel Prize in medicine is often given for work done decades before or over the span of a career, not yesterday's news.

In my view, news outlets would do well to limit the use of the word "breakthrough" and other dramatic descriptions of medical research findings. Breakthroughs are rare… and often recognized as such only well after they occur.