Clinical Advances in Hematology & Oncology
December 2014, Volume 12, Issue 12
Paul Goldberg, Editor and Publisher, The Cancer Letter, Washington, DC
H&O What are the most important traits a journalist must have in order to distinguish hype from reality?
PG From experience with mainstream media, we all know that journalists can just as easily increase hype as diminish it. In order to not hype purported advances, journalists need to follow the same principle as doctors: “do no harm.” We have many examples of journalists doing harm, often simply by not understanding the science behind a reported research finding. But to do no harm, journalists must have the background and the preparation required to parse research findings. Hype often is generated in the media by ignorance, so the stronger the background and preparation, the less likely a journalist is to hype a finding.
Journalists also need to be humble. Many journalists have a generalist background. Cancer drug development is so complex that even a journalist with a specialty in science is still a generalist, relatively speaking. If a journalist does not have the humility to ask for clarification or to crosscheck a conclusion or assertion, an article in the press can cause harm, because it does not reflect reality.
H&O Could you give any examples of when a hyped news report has caused harm?
PG A 1998 report in the New York Times about angiogenesis inhibitors has become a classic example. As well intentioned as the article may have been, it pointed to these agents as a cancer cure that was within reach. Many cancer patients, including those in the late stages of the disease, got their hopes up. The article included quotes from experts, but clearly more was needed in order to provide an accurate report of the state of angiogenesis inhibition.
Even the notion of the “War on Cancer” is an example of hype. We are not in a war with cancer, and that term should never have entered the language when it comes to covering cancer. The term did not begin with the media, although it has been hyped by the media since it was introduced. The “War on Cancer” was invented by politicians who were building their platform on a display of commitment to reducing cancer incidence and mortality through federal funding. But calling it a war hyped the effort and introduced a way of thinking about cancer that is not accurate. Cancer is not an enemy to battle in a theater of war; it is a cellular phenomenon that we need to understand through careful and persistent science, and a willingness to take incremental steps forward. But even today this erroneous term continues to appear in the press.
H&O Clearly, quoting an expert is not enough when it comes to providing an accurate story. What else should journalists do when they report on the difficult science of drug development?
PG You always can find someone to corroborate a point of view. We need to dig deeper. The first priority is to maintain a neutral stance. Perhaps the interviewee is right, perhaps he or she is wrong. It is important that journalists not have a stake in whether the expert being interviewed is correct or not. The questions asked of experts need to probe deeper. We need to ask for explanations until we fully understand the science.
This kind of investigation is exceptionally difficult. Reporting on basic science is very different from reporting on, say, research funding. A journalist covering genomics needs to discern what approach to sequencing works best in a given scenario, what assay is best, what panel of assays is appropriate for an experiment, and which predictive model should be applied. How can a journalist be expected to handle all this information?
Part of the challenge here is that journalists often are spread very thin. The same reporter may be covering sick celebrities and science at the same time. Thankfully, I do not write about celebrities with cancer, but even I can barely keep up with all the research I am trying to cover. Similarly, if a journalist’s beat is to cover multiple therapeutic areas, there is very little time to give each area its due.
H&O What are your criteria for covering a new treatment advance in The Cancer Letter?
PG First, I try to focus on findings with clinical relevance. Research at the earliest stages that is not yet applicable to clinical care is a lower priority. Abstracts are covered only minimally. I try to rely primarily on the evidence and resist any input, even press releases, from the media relations staff at pharmaceutical companies. Looking through a book of abstracts for a conference is like doing triage, assigning levels of priority to different reports.
I continue to believe that everything valuable I have learned as a reporter, I learned on the police beat. The police beat is about relying on sources. You have to be trusted and you have to trust. You have to be able to call someone just to ask about what is going on. When I was on the police beat, I would call the county jail to ask about the people who were there on a given day.
That approach is the opposite of a foreign correspondent, who travels somewhere, describes the place, and then leaves, usually never speaking to anyone in that particular place again. That is not my approach to journalism.
I may call up a researcher and ask about the work he or she is doing. Then I follow my own system of checks and balances. I will ask other sources about their take on what I have just been told. I will do everything I can to ensure I am not about to write a foolish story. This approach requires trusting people, and it also requires being careful and selective in choosing sources and interpreting what those sources say.
H&O Statistics often are used to hype results. Could you give an example of where you have seen this occur?
PG Although it is slightly less common these days, large cancer centers used to publish survival statistics to show that outcomes were better for their patients compared with patients in community practices. But a number like that is not necessarily a reflection of reality. The mix of patients at large cancer centers is different from that at community clinics. The patients at cancer centers may have more severe disease. Conversely, patients who go to large cancer centers may be healthier—they are able to travel, and they have had better health care all along—because they are well educated about treatment options. Physicians in different settings also may diagnose patients differently. The large cancer centers may be more thorough in their workups.
The only way to pierce through such a statement is with biostatistics. In particular, you need a biostatistician with expertise in cancer. No reporter can look behind the numbers on his or her own. I still see numbers every now and then in support of an institution’s claim that it provides better outcomes than community care. It may be that cancer centers have better outcomes. Personally, I believe that large cancer centers most likely do have better outcomes—but that is my belief, and no one should care about that but me. The truth is, we really do not know because the case mixes and treatment approaches differ so dramatically from site to site and there is no reliable way to adjust for these differences. You can convince yourself that you are providing better outcomes when actually you are not.
In a way, a journalist has to work more on the questions than on the answers. You have to continue asking questions and figuring out what questions to ask. If I ask many questions and trust the people who are walking me through the answers but I still do not understand the research, I do not report on it.
H&O Do you see statistics being misreported in mainstream media?
PG I think the answer to this question needs to begin with understanding the job that many reporters have. Often, a reporter is given an assignment because the topic needs to be covered, not because the reporter has particular expertise in that topic. Reporters have to be nimble, and sometimes they have to be able to wing it, because of how the job is structured.
That said, a good example of how statistics can be hyped in the media is cancer screening. Computed tomography screening for lung cancer is a fascinating story, but understanding it fully goes beyond the expertise of most journalists. When should an individual be screened? When is screening doing more harm than good, and why? What happens during screening? The “War on Cancer” paradigm promotes the idea that more screening is better. We are beating the enemy; the enemy is on the run; we are winning the battle. All of this is nonsense. What is the actual benefit of screening, and what is the harm? That is what matters.
Knowledge of statistics is not everything. When the US Food and Drug Administration’s Oncologic Drugs Advisory Committee meets to discuss new drug approvals, the same dispute arises again and again: The results of a study may be statistically significant, but are they clinically significant? Experts at the top of their profession continue to ask this question. Should journalists be expected to know the answer? The idea that understanding the fundamentals of statistics—P values, confidence intervals, and such—is the gateway to wisdom is naive. Methodology is a component of a more complex picture. Covering the US Food and Drug Administration, I learned that statisticians can take you only so far. Ultimately, a clinical decision has to be made. As journalists, we can cover the disputes, but often that is as far as we can go, and as far as we should go.
The boundaries of journalism are constantly moving. We have to balance what is feasible for us to cover in terms of the time and resources available with our level of expertise, the perspective provided by our sources, and of course the data itself.
H&O What issues are currently on the horizon that you see as being susceptible to media hype?
PG The proliferation of molecularly guided therapies is an area of concern. Doctors may be determining the best treatment based on a test that may not be validated, leading to misuse of drugs. There is no way to track this occurrence. The science behind the tests is extraordinarily complicated, but I think it is one of the most important issues in medicine today. Coverage of this area is one of my current areas of interest. There is a new breed of charlatan out there: a scientist who insists that the science behind a test is solid, and then proceeds to treat patients based on molecular findings rather than on practice guidelines.
Journalists reporting on this area may rely on experts, but which experts? A reporter constantly has to ask, “Am I crazy?” and “Am I seeing things, or are these results real?” I never assume that my conclusions are correct. In many ways, a journalist parsing research is the same as a peer reviewer reading a study. Peer review is an art form; it is extraordinarily difficult and requires the utmost humility. Distinguishing hype from reality in journalism is no different.
Suggested Reading
Brawley OW, Goldberg P. How We Do Harm: A Doctor Breaks Ranks on Being Sick in America. New York, NY: St Martin’s Press; 2012.
Goldberg P, Conti RM. Problems with public reporting of cancer quality outcomes data. J Oncol Pract. 2014;10(3):215-218.
Kolata G. Hope in the lab: a special report—a cautious awe greets drugs that eradicate tumors in mice. New York Times. May 3, 1998:A1.
Goldberg P. Are cancer drugs getting better? FDA’s Richard Pazdur says, “yes.” The Cancer Letter. February 14, 2014. http://cancerletter.com/articles/20140214. Accessed October 28, 2014.
Ong M. Harvard physician, whose cancer was spread through morcellation, seeks to revamp FDA regulation of medical devices. The Cancer Letter. July 4, 2014. http://cancerletter.com/articles/20140704. Accessed October 28, 2014.
Goldberg P. “Wild west” of molecular testing? Caris engaged in aggressive marketing, improper Medicare billing, lawsuit alleges. The Cancer Letter. August 8, 2014. http://cancerletter.com/articles/20140808. Accessed October 28, 2014.