Valuing Negativity
Ben Goldacre’s most recent Bad Science section of The Guardian has a thought-provoking discussion of the relative ease of publication and degree of press coverage devoted to positive results in science, as opposed to negative results.
I expect that it is not particularly surprising to anyone that the media generally report sparsely, if at all, on negative results, while reserving significant acreage, hyperbole and optimism for new claims, however speculative. It is equally easy to identify the broad reasons why this happens. From the researcher’s side, one is often much more excited about new proposals to explain unsolved problems and therefore more likely to talk about them more. I also imagine that it would be much more difficult to persuade the Public Relations departments at any university to publicize a new negative result to local and national media outlets.
From the media’s side, there are fewer and fewer resources for science reporting and, indeed, fewer and fewer science reporters to get the job done. A tightly stretched science reporter is going to find it much easier to find stories about positive results than negative ones, and will have a more straightforward time finding scientists willing to give their time to discuss such a result.
Finally, it’s clear that the public is much more likely to find a positive result or speculation more interesting; and who can blame them really?
However, Goldacre’s column also points out that such bias against negative results is not confined to newspapers or television, but that it also extends to professional scientific publications. Admittedly, his focus is mostly on medicine and the extent to which trials contesting highly positive claims about a drug typically receive far less attention than the original papers.
Major academic journals aren’t falling over themselves to publish studies about new drugs that don’t work. Likewise, researchers get round to writing up ground-breaking discoveries before diligently documenting the bland, negative findings, which sometimes sit forever in that third drawer down in the filing cabinet in the corridor that nobody uses any more. But it gets worse. If you do a trial for a drug company, they might – rarely – resort to the crude tactic of simply making you sit on negative results which they don’t like, and over the past few years there have been numerous systematic reviews showing that studies funded by the pharmaceutical industry are several times more likely to show favourable results than studies funded by independent sources. Most of this discrepancy will be down to cunning study design – asking the right questions for your drug – but some will be owing to Pinochet-style disappearings of unfavourable data.
But I think such an attitude exists in a much wider scientific context and that it is a bit of a shame.
There have been a few times during my career when I’ve written a (what I, naturally, consider to be very cute) negative paper. Now, I personally have no beef with how I’ve been treated by scientific journals, so don’t take this as a complaint; but it has been clear to me that referees and the journals themselves tend to be much less excited about such papers. In fact, referees sometimes comment that they don’t really consider negative results sufficiently interesting. I’ve heard similar complaints from a number of colleagues over the years and I must say that I find related thoughts going through my mind when faced with refereeing a negative paper (although I do my best not to act on them).
I think that this kind of bias against negative results is a shame because they play a remarkably important role in the progress of science.
There are a number of ways in which this is true. First, there is the rather obvious statement that if one can demonstrate cleanly that a given idea or set of ideas is inconsistent or at odds with an established piece of data, then one provides an invaluable service to science, pruning the tree of speculations. Good science indeed!
Second, it is perhaps less well known outside the physics community that theorists look on the strongest negative results – the decisively-named no-go theorems – as distinct challenges to their physicisthoods (OK, not really a word, but it should be). When something gets called a theorem, it means that it starts with some clearly expressed assumptions from which the deathblow result then logically follows. Those juicy assumptions are just asking for it in the eyes of aggressive young physicists.
Perhaps the best-known example of this is the story that led to the idea of supersymmetry. There is a famous and beautiful theorem known as the Coleman-Mandula theorem, after it’s discoverers – Sidney Coleman and Jeffrey Mandula. Titled All Possible Symmetries of the S Matrix, and published in 1967, it has a great negative paper first sentence of the abstract
We prove a new theorem on the impossibility of combining space-time and internal symmetries in any but a trivial way.
The basic point is that if one assumes that the generators of the internal symmetry group are commuting operators (and that their commutation relations define the group – i.e. that they comprise a Lie algebra), then the only possible total symmetry is a direct product of the space-time symmetries (the Poincaré group) and the internal symmetry group. This is what they meant by trivial in the abstract.
If this had been the end of the story, then bosons and fermions (and therefore force carriers and matter) would be destined to forever remain distinct. But here comes the loophole. The 1975 Haag-Lopuszanski-Sohnius theorem (after Rudolf Haag, Jan Lopuszanski, and Martin Sohnius) pointed out that if one relaxes one of the assumptions, and allows anticommuting operators as generators of the symmetry group, then there is a possible non-trivial unification of internal and space-time symmetries. Such a symmetry is called supersymmetry and, as you know, constitutes a large part of current research into particle physics.
No-go theorems are fun in physics because they formalize where the important barriers lie and provide guidance about the directions of future attacks on the problem in question. Negative results in general, although not quite as glamorous or exciting, are still great stuff. We should celebrate them. Plus, we don’t want to be like the medical community do we?
-
http://electrogravity.blogspot.com/ Science
-
Thomas Larsson
-
http://insti.physics.sunysb.edu/~siegel/plan.html Warren
-
http://blogs.discovermagazine.com/cosmicvariance/mark/ Mark
-
http://countiblis.blogspot.com Count Iblis
-
Steve
-
http://insti.physics.sunysb.edu/~siegel/plan.html Warren
-
http://muon.wordpress.com/ Michael
-
http://motls.blogspot.com/ Lubos Motl
-
Chris W.
-
Supernova
-
Q
-
http://blogs.discovermagazine.com/cosmicvariance/mark/ Mark
-
Matt
-
Elliot
-
Richard
-
http://www.haloscan.com/comments/59de/115017290815882577/?a=26669#197784 Plato
-
Q
-
http://electrogravity.blogspot.com/ Science
-
http://blogs.discovermagazine.com/cosmicvariance/mark/ Mark
