Another Argument for Quality Science Journalism

By Mark Trodden | June 21, 2008 12:42 pm

We’ve written on a number of occasions here (and elsewhere) about the importance of good, reliable science journalism. Now, writing in his Bad Science column in The Guardian, Ben Goldacre points to an extremely concrete reason, with important ramifications for society, to encourage quality science writing.

There are all kinds of reasons to be concerned about and interested in science journalism. Are important issues getting the right amount of coverage? Is there a reasonable balance to the stories? Do journalists rely too much on the opinions of a few friendly experts, or do they seek out diverse expert views? Do university press releases drive coverage rather than inform it? Do funding agencies pay to much attention to research and authors who receive press coverage? I could go on and on. Some of these issues are mostly of interest to academics, but to some extent they all directly affect not only the public’s understanding of current scientific progress, but also their ability to make an informed decision to support (or not support) future scientific endeavors.

But there is another, more direct reason to hope for strong coverage of science. Goldacre reports on several studies showing that when it comes to health issues, the public pays a remarkable amount of attention to press coverage of a given question. Among other things, he notes that

A 2005 study in the Medical Journal of Australia looked at the impact of Kylie Minogue’s breasts on mammogram bookings. They rose by 40% during the two-week publicity peak, and six weeks later they were still up by a third. The increase among previously unscreened women in the 40-69 year age group was 101%. These surges were unprecedented.

…A systematic review from the Cochrane Collaboration found five studies looking at the use of specific health interventions before and after media coverage of specific stories, and each found that favourable publicity was associated with greater use, and unfavourable with lower.

One might think that this is an extremely positive thing, since to a large extent the public seems to trust scientific results, and holds science is high regard. However, the main point of Goldacre’s column is to discuss a recent analysis, by former journalist Gary Schwitzer, of 500 mainstream media health articles from the US.

The results were dismal. Only 35% of stories were rated satisfactory for whether the journalist had “discussed the study methodology and the quality of the evidence”: because in the media, as you will have noticed, science is about absolute truth statements from arbitrary authority figures in white coats, rather than clear descriptions of studies and the reasons why people draw conclusions from them.

Only 28% adequately covered benefits, and only 33% adequately covered harms. Articles routinely failed to give any useful quantitative information in absolute terms, preferring unhelpful eye-catchers like “50% higher” instead.

So it turns out that what the public really trusts is whatever journalists tell them about science. This is why it is so important to do everything we can to support good science journalism, and to resist the temptation to contribute to poor efforts by overly sensationalizing our own work when speaking to journalists about it.

But it isn’t easy, given the juicy data the article also contains about how New York Times coverage skews citations!

  • Christy Caldwell

    I want to plug a great program at University of California Santa Cruz. They offer a graduate certificate program in Science Communication. All of their graduates have science degrees, many of them Ph.D. level.

    http://scicom.ucsc.edu/SciWriting.html

  • http://backreaction.blogspot.com/ B

    Do you happen to know whether there are more and/or more recent studies on how newspaper coverage influences citation behavior in academia?

  • http://blogs.discovermagazine.com/cosmicvariance/mark/ Mark

    Not off the top of my head B. Sorry!

  • Mike

    Mark,

    The problem is that not even real scientists can evaluate studies outside their own area of expertise (let alone science journalists). I call this the “Nova Syndrome:” I can watch an episode on, say, the Anasazi (my area) and notice a few half-truths, a few oversimplifications, and a few inappropriate people interviewed, even when the overall show is quite good. I have to assume you guys would have similar comments on Nova shows about, say, String Theory or the Big Bang; but I forget to be critical, and eat it up. I simply don’t know better (although I’m learning, in part because of this website and Sean’s Great Teaching gig).
    You hit the nail on the head with your suggestion that we all try to be less sensational. But there will probably always be those among us who like to see their names in the paper. Note that the media never present refutation of claims that were overstated or premature.

  • http://blogs.discovermagazine.com/cosmicvariance/mark/ Mark

    That’s a problem Mike, but not the one I’m discussing here. Certainly “a few half-truths, a few oversimplifications, and a few inappropriate people interviewed” can be a problem (although sometimes part of that can be justified). But the lack of presentation of the scientific method, and the choice to present science as black and white in frontier areas where details and interpretations are still being sorted out are decisions that apply across areas and we can recognize even in areas far from our own.

  • Ijon Tichy

    So it turns out that what the public really trusts is whatever journalists tell them about science.

    If people are so gullible as to trust journalists, and so indifferent as to not learn science and the methods (note plural) of science from textbooks, why should we care about the quality of science journalism? If you’re worried about the future of (taxpayer funded) science, then what you really want from science journalism is quantity, entertainment, and propaganda. Know your audience.

  • http://blogs.discovermagazine.com/cosmicvariance/mark/ Mark

    The reason we should care is that quality science journalism can help engage people in the scientific method and help them to develop critical thinking and an understanding of what science is about. Then perhaps they would be less gullible.

    One can be worried about multiple things at once if one really thinks hard.

  • anonymous pot-stirrer

    Assume for a moment that you are a scientist who finds himself with a fabulous (according to your peers) new theorem/formula/find that is starting to attract media attention. Further assume (and this may be hard for some of you) that you liken the need to talk to the press with the need to have a dislocated shoulder set – it’s gotta be done, and you’ll just have to close your eyes, bite down on something, and hope you don’t pass out from the pain.

    Who would you want to be interviewed by, and why? Who would best be able to accurately represent your theorem/discovery without sensationalizing it? Who would strike the best balance between talking about you and talking about your science?

  • Ijon Tichy

    I certainly don’t believe that quality science journalism makes adults less gullible, helps them develop critical thinking skills, and leads them to gain an appreciation of science and its methods. For the vast majority of adults, it’s too late for those things to happen. Children are another matter, but how are they to filter out the poison dished out to them by their magical thinking parents and teachers? Only a revolution in education and child-rearing methods can bring about the change you desire. And that will not happen in a country where the free market is worshipped like a god, and government only serves the rich.

  • http://swimmingthechannel.blogspot.com Clara

    I think the first thing on the agenda should to pass some sort of law disallowing phrases such as “major breakthrough” and “revolutionizing [field name].” And probably just the word “prove” altogether, eh?

    ;-)

  • http://blogs.discovermagazine.com/cosmicvariance/mark/ Mark

    Ijon Tichy. Clearly you and I disagree. I certainly am going to continue to call foul on poor science journalism and celebrate excellent examples, while working in other ways to bring about better education, and better government. There are lots of things that need attention in society, and I tend to take the attitude that the more informed and involved citizens we have, the better the chances of progress.

    One thing is certain, sitting back and complaining about the government/system is unlikely to help.

  • John Merryman

    Journalism presents as clear a picture of science as science presents of nature. It is a slow climb out of the muck and idealism, whether of science, journalism, politics, religion, or whatever, is still subjective.

  • http://backreaction.blogspot.com/ B

    Hi Mark,

    Well, if you come across something let me know.

    I am with you on the problem of science journalism, this topic touches on a much more general issue that is that people don’t want to realize one can’t run science like a business. Letting similar tactics influence the scientific community is going to be a huge disaster, in fact it already is. People have started to pay a lot of attention to how they advertise and sell their work, this inevitably goes on the expenses of clarity and integrity, not to mention that it is an investment of time an effort that is wasted into work other than research. Similarly, science journalists – though I am sure most of them have a genuine interest in the topic – have the pressure to make stories that sell, and this goes on the expenses of content which is excused with the ‘system’, i.e. one has to get the money in, do what the editor wants, please the audience which likes superficial and entertaining stories. Their responsibility to report accurately just isn’t a variable that is measured in money in any reasonable way. To put it differently: the micro-interest don’t yield a desired macro-result. That’s a problem. And it’s a completely obvious problem. So why then are we so unable to change anything about it?

    Best,

    B.

  • http://name99.org/blog99 Maynard Handley

    “The results were dismal. Only 35% of stories were rated satisfactory for whether the journalist had “discussed the study methodology and the quality of the evidence”: because in the media, as you will have noticed, science is about absolute truth statements from arbitrary authority figures in white coats, rather than clear descriptions of studies and the reasons why people draw conclusions from them.”

    Come on, Mark, this is just silly, yet another manifestation of scientists’ wish that everyone be a scientist.

    When you read an article about a new historical discovery, do you expect to read an in-depth discussion of the historical methods used? Do you expect to see addressed the on-going controversy between the camp who claim that roman tax records can be taken at face-value vs those who claim they were routinely under-estimated, each marshalling its obscure items of evidence (presented, of course, in the original Greek, Aramaic and Egyptian)?

    Even if you are a scientist, if you are not a specialist in the field, what good will this information do you? When I read an article by economists claiming that the effect of a cut in the gasoline tax will be …, I judge it based on the individuals involved, their political affiliations, what they have said in other contexts and whether they’ve been right, what is said by other commentators I respect and so on. Sure, sure, I could devote two years of my life to an in-depth study of the material after which I could make my own informed decision. But I don’t have two years to spend on this project, along with also spending time to validate claims about climate science, nanotechnology and the latest archeological discoveries. At some point the only thing to do, I don’t care how intelligent, educated and fast you are, is to trust other people.

    So where does that leave us? The media occasionally report mistaken medical results, but that’s because medicine is a hard subject to do research in. What exactly would change if the story included a note saying “these results are provisional and it’s not quite clear the extent to which they are correct”? The people who want to believe will still believe, the people who don’t want to will not; just like now.

    Sure articles could present a paragraph of the specific details of the results, presented in a scientifically rigorous form, and that would probably make the world a marginally better place. But in the grand scheme of problems on earth, it just doesn’t strike me as a very big deal. The willful unwillingness of scientists to accept the reality of the world that I presented in my 2nd and 3rd paragraphs strikes me as a far larger, far more serious problem.

  • http://name99.org/blog99 Maynard Handley

    “I certainly don’t believe that quality science journalism makes adults less gullible, helps them develop critical thinking skills, and leads them to gain an appreciation of science and its methods. For the vast majority of adults, it’s too late for those things to happen. Children are another matter, but how are they to filter out the poison dished out to them by their magical thinking parents and teachers?”

    Ok, Ijon, we’re talking science here.
    Please provide for me the scientific evidence (as opposed to romantic Rousseuian fairytales) that the evils of adults are due to the perversion of nice kindly children?

    So children are curious about the world? They also believe everything anyone tells them, have almost no empathy, and can be astonishingly cruel. They also don’t engage in any sort of sexual activity because, as you may have noticed, their bodies (and apparently their minds) change a whole lot during their teen years — a change that has nothing to do with the parents and adults around them.

    This melange of assorted factoids does not in any way add up to any sort of proof that “raising them right” would turn them all into scientists, or at least rational sceptics, when they become adults.

  • http://blogs.discovermagazine.com/cosmicvariance/mark/ Mark

    Maynard, there is a huge difference between asking that science writing provide us with the same level of subtlety that the actual discussions between scientists contain, and asking that they try to convey that frontier issues are complex, unsettled and in flux, even when discussed by reputable experts.

    And yes, when I read articles in fields far from my own technical expertise (politics and economics spring to mind), I am frequently disappointed at them for precisely this reason.

    So I’m afraid you’re going to have to do a lot better if you want to justify your use of the word “silly” in your comment.

  • Ijon Tichy

    One thing is certain, sitting back and complaining about the government/system is unlikely to help.

    Well I certainly agree with that. Which gets back to the difficult question of what can one man or woman do? A topic for another day I guess.

  • Ijon Tichy

    Ok, Ijon, we’re talking science here.
    Please provide for me the scientific evidence (as opposed to romantic Rousseuian fairytales) that the evils of adults are due to the perversion of nice kindly children?

    Looks like you’ve already created your own narrative (e.g. “Rousseauian fairytales”, “nice kindly children”, “children are curious”, “evils of adults”), so I doubt if anything I write will do you any good. I could list hundreds of references on the subject of the long-lasting effects of childhood maltreatment, from books to reviews, from scientific papers to advocacy reports. Instead, and for the benefit of everyone here, I provide a link to one particular study (PDF, 345 KB), which conveniently provides a nice review of the subject as well.

  • Proteus

    Maynard:

    You make a valid point that the general public needn’t all absorb every detail about a particular study or subject. However, I’m going to guess that “satistactory” doesn’t mean “an exposition of the history of economics”, but rather that they present any facts at all. They don’t need to give a specialists’ account, but they have to tell us something about what was done. If 90% of the articles were providing this information, the public would, in my opinion, begin to wonder why the other 10% withheld the information, and the public might begin, subconsciously even, to understand what science is about.

    Ijon:
    You forgot Bush [as a cause of any ills in society].

NEW ON DISCOVER
OPEN
CITIZEN SCIENCE
ADVERTISEMENT

Discover's Newsletter

Sign up to get the latest science news delivered weekly right to your inbox!

Cosmic Variance

Random samplings from a universe of ideas.

About Mark Trodden

Mark Trodden holds the Fay R. and Eugene L. Langberg Endowed Chair in Physics and is co-director of the Center for Particle Cosmology at the University of Pennsylvania. He is a theoretical physicist working on particle physics and gravity— in particular on the roles they play in the evolution and structure of the universe. When asked for a short phrase to describe his research area, he says he is a particle cosmologist.

ADVERTISEMENT

See More

ADVERTISEMENT
Collapse bottom bar
+

Login to your Account

X
E-mail address:
Password:
Remember me
Forgot your password?
No problem. Click here to have it e-mailed to you.

Not Registered Yet?

Register now for FREE. Registration only takes a few minutes to complete. Register now »