Science: Growing Too Fast?

By Neuroskeptic | September 30, 2012 11:05 am

There’s a widespread perception among scientists that we’re living in an era of relentless growth in terms of the number of scientific papers being published.

Many say that quantity has increased at the expense of quality: people are publishing “any old rubbish” or splitting their work into as many papers as possible, driven by the publish-or-perish culture of modern academia.

But is this true? To try and find out, I looked at the number of papers published each year, in English, on PubMed, for the past 30 years.

Here’s the data: it shows an increase in the number of papers coming out each year, except for a small negative blip around the year 1997:

Now, when I first eyeballed this curve, I got the impression that growth has accelerated recently, consistent with the “recent pressure to publish” idea.

But here’s the same data with each year’s publications expressed as a ratio to the previous year’s:

This reveals that the relative annual growth in the number of papers published has actually been pretty constant over the past 30 years. It’s generally been around 4% (ratio of 1.04), and almost always within the range 2% to 6%. In other words, every year, scientists publish the same number of papers they did last year, plus about 4%.

The past few years have not seen especially strong growth, relatively speaking. At most we can say that year-on-year growth has been at the upper end of the historical range, 4 to 6%, but that’s no faster than in the 1980s.

Still, this is a lot of growth. Assuming that it stays at 5% year on year, we’d expect a million new papers published in 2016, and two million in 2030.

But is that really feasible? Is there any good reason that science should grow exponentially in this way? Can that continue, or will we reach “peak science” or at least a plateau?

CATEGORIZED UNDER: graphs, history, papers, science
  • http://www.blogger.com/profile/06834641501438709866 Jan Moren

    Compare to the number of active researchers worldwide perhaps? Outside the old stalwarts in US and Europe there's a steady stream of up and coming countries that are establishing their own research programs and building up competence. I wouldn't be surprised if this gradual increase corresponds fairly well to the total worldwide increase in research resources.

    And if that is the case, don't expect it to slow anytime soon. There's many countries, including most of a continent, that have still to embark on this particular endeavour. Most of them will, at one point.

  • http://www.blogger.com/profile/06647064768789308157 Neuroskeptic

    That's surely part of it, yes; the problem is it's hard to estimate the number of 'researchers' especially in bio/medicine where clinicians increasingly publish research as well.

  • http://www.blogger.com/profile/06778095258574983825 Neuro Polarbear

    The real question is how much growth we see relative to the rise in human population. If there are more people, there should be more scientists and more science, and that's reasonable.

  • http://www.blogger.com/profile/00890404304081225894 StokesBlog

    Without question, the more science the merrier!

    But any boom raises a serious management problem. Search engines are getting better at organising this booming literature as a whole, but there are still far too many potentially relevant papers too read. Moreover, if we are to redress publication bias (publish more null effects and replications), this is only going to increase the burden of keeping up with the literature.

    It would probably make sense to abandon the current manuscript format for 90% of experiments, and report the results in more of a database format. Particularly novel, exciting, etc (i.e. scientifically important, however defined) could be given the opportunity for a more detailed and theoretical report.

    Keep the science growing!

  • omg

    Science has become a way of life much in the way religion had. It'll decay when it decentralizes into the public sphere much in the way monks/priests don't write scrolls 14 hours at a time anymore. Religion has churches I wonder where science would flourish. I'm guessing something genetics, neuroscience related centre of somesort in every neighbourhood. Reprogram your mind, change your destiny.

  • http://petrossa.wordpress.com/ petrossa

    It's indeed becoming a data glut. Reading papers becomes more filtering out the flotsam for the few pearls.

    Which is consistent with the law of diminishing returns.

    Funny how such a simple economic principle works out all over the place.

  • omg

    That movie Gattaca was depressing it didn't make sense.. they were still typing on a box. I'm thinking more Hunger Games the rich getting richer, the poor marginalized even more. More mindwashing with the yeti, loch ness monster etc. becoming a reality. Everything would be an illusion except time.

  • Ivana Fulli MD

    petrossa,

    My (poor) understanding of the law of diminishing return is that, in agriculture for example, after all the good fields have been taken some people are obliged to cultivate bad ones and produce poorer harvests.

    I fail see how it applies here. Please explain for the lay people in economy.

  • http://www.blogger.com/profile/02537151821869153861 Andrew Oh-Willeke

    One might compare the rate of paper growth the the growth in the number of people with doctoral level degrees in medical and biological science fields, which would be a pretty good proxy for the number of researchers.

  • Anonymous

    Hmmm….don't think the growth of science in and of itself is a problem, as much the structure that's set up to deal with it – e.g., professorships, the publication process, etc. This was bound to happen at some point. As StokesBlog pointed out, traditional methods of publication will likely have to go at some point.

  • http://www.blogger.com/profile/00187465138890222167 LokaSamasta

    Was the growth slower before researchers started using computers heavily?

    Maybe the growth can be explained by copy and paste, and spreadsheets ;)

  • Vince

    In agreement with Anonymous's comment, the velocity itself isn't a problem, but rather the burden it places on the structures and hierarchy that supports it. These are systems that are filled with huge legacy and bureaucratic inertia and turn extremely slowly.

    Our EE friends have gone through a somewhat analogous period of compounding growth and they've survived Moore's law, but done so by being quite nimble and open to adopting new IT tools and technologies to partition work, share information, etc.

    I believe it was Intel's Andy Grove who I heard make the comment that during the design of the 8080, everyone knew the floorplan. Today, with a 2 Billion transistor Core or Tukwila, there might be one or two. Might.

    It's getting harder and harder to keep up with it all. Unless you read neuroskeptic, of course!

  • Anonymous

    You could try to run a program to filter through authors and removing duplicate names to see if the increase in researchers coincide with the increase in papers. It could also be a relationship of faster access to information. Since researchers can find if they've been scooped much sooner than before they would use less time attempting to publish something hat has already been done.

  • http://brainsidea.wordpress.com/ brainsidea

    I somehow like Casey Bergman's take on this. She reviews Derek de Solla Price's 'Little Science, Big Science' (1963) about the same trends discovered in your post in a wider historical perspective.

    For example, as opposed to Neuro Polarbear's suggestions, the doubling time of the human population (50 years) is much longer than the doubling time of the scientific output (10-15 years). So, human population growth cannot be the full explanation.

    More than that, scientific output cannot outpace population growth forever. Thus, “either everyone on earth will be a scientist one day, or the growth rate of science must decrease from its previous long term trends. [de Solla Price] then goes on to argue that the most likely outcome is the latter, and that scientific growth rates will change from exponential to logistic growth and reach saturation sometime within 100 years from the publication of his book in 1963.”

    From this perspective, the trends you are describing are real, albeit only the signs of a period of transition.

    Link:
    http://caseybergman.wordpress.com/2012/08/26/the-logistics-of-scientific-growth-in-the-21st-century/

  • infinidiv

    I thought I would do a quick fact-check to compare doctoral degrees awarded versus papers published. The easiest way to do this was to base it on US numbers as these are the easiest to get a hold of (it would great if others started adding stats for more countries here!)

    …………….PhDs Awarded…Articles Published
    1999-2000…..106,494……638,512
    2009-2010…..140,505……951,447
    Increase………….32%……….49%

    Thats a pretty big difference!
    Although only based on the US, I think that many countries with a greater increase in publication numbers also also have a greater increase in new PhDs awarded (such as China and India with much faster growing middle classes and academics, by percentage). I think the US is one of the more stable examples over this time period.

    References
    PhD Stats: http://nces.ed.gov/fastfacts/display.asp?id=72 [based only on overall doctorates awarded]

    Publication Stats: scopus.com [only limits set on search were country of affiliation (US) and years (as stated above)]

    of course this does not account for the fact that many papers will have international affiliations, but I am not sure how to account for that, unless someone finds credible global statistics for doctoral degrees awarded…

  • Anonymous
  • http://www.blogger.com/profile/06647064768789308157 Neuroskeptic

    Thanks for the great comments.

    infinidiv: That's interesting and consistent with the idea that any given researcher is publishing more nowadays; but another interpretation is that there are more publications from non-PhDs nowadays. And although I don't know any figures, anecdotally, there is a pressure nowadays for people who don't desire a career in research to publish a few papers to help their clinical or other non-scientific careers.

  • http://petrossa.me/ petrossa.me

    @ivana

    You don't see the parallel?

    The law of diminishing returns states that in all productive processes, adding more of one factor of production, while holding all others constant , will at some point yield lower per-unit returns. The law of diminishing returns does not imply that adding more of a factor will decrease the total production, a condition known as negative returns, though in fact this is common.

    More papers doesn't equally yield more results, either because you can't find them due to overkill, or due to publication pressure more sloppy papers.

  • Ds

    A lot of crap is published. The question is: What is the growth in crap?

    Are we close enough to the limits of human science that we should expect a growing fraction of crappy papers if the number published papers stays constant.

  • AStatsSkeptic

    Is this not the usual problem of selective figures being presented without sufficient context? In the wrong hands it does indeed nicely play to an alarmist agenda and could be presented as “the inflation rate of papers per scientist is 4% per annum” in a shock horror revelation. But I am really not convinced by this. I would argue there may be nothing here of significance to explain or indeed the interesting thing is that productivity may actually be going *down* per person. We just can't tell solely from the graph data presented.

    Firstly, as others have pointed out the graph does need to be normalized in some way over the time period of 1970-2011 for it to be interpretable. In this period the world population has increased from about 3.7 billion to about 7 billion. Not at a steady year-on-year rate, but at an average of about 1.5% per year over the period. Link:
    http://en.wikipedia.org/w/index.phpitle=File:World_population_growth_rate_1950%E2%80%932050.svg&page=1

    Secondly there is no reason to suppose that the proportion of publication active researchers will be in line with overall population demographics. Globalization means that more people are getting higher degrees per year and publications are now coming from not just PhDs but MDs (for whom there is an increasing cultural drive to publish clinical findings), therapists, practioners, commercial companies and a whole set of parties with a vested interest in promoting their research. It seems plausible (but without data one cannot be sure) that there are more *people* publishing articles year on year.

    Thirdly, the sampling from Medline is for “English Language papers”. This alone may have introduced a specific bias. The other consequence of industrialization over the past 40 years (besides rising numbers of researchers overall) is specifically the rise of English as the global academic language for publication. Similarly with freer access to journals from the former Soviet Union and Iron Curtain countries and from other previously impoverished countries. These factors may make it appear that more publication is going on overall when really it is more publishing going on *in English* (in mainly Western journals).

    Fourthly, there may well be another specific bias derived from using an electronic source to compile the graph numbers which further inflates the calculated paper growth ratio. I imagine it is much easier for Medline to access all modern journals (from 21st century) as they will be inevitably be available in electronic format. I am not convinced that they have done such a good job for the paper journals of 1970 as not all may be available digitally.

    Unfortunately we do not know the real figures for the discussion points above. My first point above has some evidence to suppose a correction of approximately 1.5% annum in the yearly paper growth ratio is warranted. But suppose points 2, 3 and 4 each further depreciated the ratio by an additional 1% per annum. Each alone seems like a modest but plausible correction. Then overall we would need to decrease the year-on-year growth by 4.5%.

    Et voila, we now have a *corrected* graph of English language papers published per year (per person) of -0.5%. Oh no. Productivity per person is going down per year. We are surely doomed!

    Ah. The fun of stats.

  • http://www.blogger.com/profile/06647064768789308157 Neuroskeptic

    Interesting points, a few thoughts:

    Normalizing to population growth might not be appropriate because the great majority of papers come from countries with low pop growth and vice versa. It seems odd to “correct” for the growing population of Africa & India, given that per head of population they haven't published much, although that is now changing.

    The English language bias and the electronic source bias are good points though and ought to be possible to measure by making use of clever searches, if I have time I'll look into those.

  • http://www.blogger.com/profile/16203083806436919715 Bernard Carroll

    As Ds put it, “A lot of crap is published. The question is: What is the growth in crap?”

    To answer that we need to factor in another variable – impact. Off the top of my head, the median number of citations per published paper is zero, while the arithmetic average is 2. So, NS, you might try to interrogate the Thomson Reuters citation data base to see the temporal trends in high impact papers. They will be quick to point out that the operational definition of high impact differs by field. Just the same, you can probably come up with an estimate.

  • Anonymous

    Just because something is/isn't something doesn't necessarily make it crap/good though.

  • http://www.blogger.com/profile/04971702402040080054 Jen Daisybee

    This is an interesting concept to ponder. I just wanted to say thanks for the thought-provoking post. I always enjoy your blog.

  • http://www.blogger.com/profile/16115218690707131186 cabbagesofdoom

    I certainly wish that (1) more emphasis was placed on quality rather than quantity; (2) quality was assessed more on actual scientific merit and not the impact factor of the journal; (3) high impact factor journals cared more about scientific rigour and less about novelty.

    The current setup definitely encourages scientists to rush publications before they are really ready and this is definitely filling up the literature with results that cannot be reproduced and are probably wrong. I certainly don't trust anything I read in the literature unless I know and trust the authors (not necessarily first degree) or it has been independently verified.

    It's annoying but it's (hopefully) not going to kill science, just slow it down. (Which is ironic, really, as I think the problem stems from a desire to speed it up. More haste, less speed…)

  • EvilGuru

    I think one should be very careful about agreeing with the consensus that more publications somehow must mean more 'bad' publications.

    There seems to be some deep rooted idea that 'less is more' and that only a finite number of elite people have special insight to publish 'good' papers. And that any other other growth in papers must come from salami slicing of 'good' data from mediocre researchers and 'crap' data from 'poor' researchers. Is it only me that thinks that is a very arrogant view?

    Personally, I suspect that this mindset will turn out to be bunk. I agree with Neuroskeptic, and would go further, to the extent that ALL information is useful – bad and good. I welcome the growth of open publishing (if it meets technical and formatting requirements even though reviewers do not 'like it' it should be published).

    Publish everything. Truth will out.

  • Anonymous

    Just to pick up on AStatsSkeptic's comments:

    (1) I agree the graph is misleading because the more informative metric would be the papers per person per year, rather than “raw” papers per year.

    (2) The relevant normalization factor is probably the number of people in the world who are “research active” as in those people expected to publish research papers as part of their job. That will not be the same as the global population growth. It is probably roughly related to global PhDs, but this could either be an under or an over estimate.

    (3) I agree that measuring paper growth through Medline may introduce specific biases. The growth of English as the language of science must have had an effect.

    Rather than focusing on raw figures it would certainly be interesting to know what the variation of publications per person is over the years. I think this would be very revealing for this discussion.

  • John D.

    A lot of the industrialized world has grown at least 4% per annum on average over this period. This probably correlates with global fridge sales too.

    4% annual growth in scientific papers seems fairly modest. Is this really a cause for concern? As a psychologist, I don't a perceive a problem with the publication output. Rather, I see a problem with people thinking that an increase in publication must inexorably mean a watering down of quality. It as if there is some underlying idea that there is a finite amount of new information that can be “produced” per year. And if that boundary is exceeded then clearly the excess publications must be “crap”.

    Flawed reasoning I think.

  • http://petrossa.me/ petrossa.me

    @John D
    Even assuming that x% of papers published are not that good is a constant then still with growth there are more not that good papers.

    And even if there is actually an overall improvement in quality, the quantity makes it harder and harder to find the gems, or in fact papers that don't kick in open doors/confirm what we knew already/study some weird abstract anomaly

    Reading out of pure desire to learn, and being free since about 15 years now i can read much more then your average working professional.

    Trust me that in general since the last 10 years quality got lower and quantity got higher.

  • Ivana Fulli MD

    udingT-petrossa 1 October 2012 18:42,

    Thanksfor pointing out my confusing the law of diminushing return with negative return :

    ///The law of diminishing returns states that in all productive processes, adding more of one factor of production, while holding all others constant , will at some point yield lower per-unit returns.

    (…)
    More papers doesn't equally yield more results, either because you can't find them due to overkill, or due to publication pressure more sloppy papers.///

    It has been a long time since I have thought that a few educated clients must be part of the selection of papers for publication in the best medical journals- be ti only to assess the right priority of topics.

    I now sincerely think that we need fiancial analysts too in the reviewing processes of medical science articles to be published.

    ///

  • John D.

    @petrossa.me

    “Trust me that in general since the last 10 years quality got lower and quantity got higher.”

    Sure, I feel that way too. From my perspective I also see more papers being published (higher quantity) and in those I see a fair few of questionable value (lower quality).

    What I can't be sure about is whether the proportion of crap vs total is changing over time or not. That would the interesting metric to get at. Although it feels that the ratio is going up it may be a perceptual bias. If the ratio were to stay roughly the same we would still see more crap papers this year compared to last year (based on the overall growth rate in papers). But we'd perceive this as an escalating rise in crappiness.

  • http://www.blogger.com/profile/06647064768789308157 Neuroskeptic

    John D: Right. Also, the % of crap papers could increase even if the % of crap science didn't, supposing that people nowadays, rather than publishing a few papers of their best science, published everything they did, bad and good.

  • http://www.blogger.com/profile/09905763419815019881 Wilson

    There is nothing particularly bad about publishing good and bad papers, as has been mentioned above, I think it is good that a wide variety of science can be published for the world to bear eyes on. What is bad though, is that we hardly have the infrastructure to be able to support such a large amount of papers (as has been said). With Google Scholar being one of the primary research points for many people, this is somewhat alarming. It is hardly perfect…

  • Patty V

    What’s the author indicating?

NEW ON DISCOVER
OPEN
CITIZEN SCIENCE
ADVERTISEMENT

Discover's Newsletter

Sign up to get the latest science news delivered weekly right to your inbox!

Neuroskeptic

No brain. No gain.

About Neuroskeptic

Neuroskeptic is a British neuroscientist who takes a skeptical look at his own field, and beyond. His blog offers a look at the latest developments in neuroscience, psychiatry and psychology through a critical lens.

ADVERTISEMENT

See More

@Neuro_Skeptic on Twitter

ADVERTISEMENT
Collapse bottom bar
+

Login to your Account

X
E-mail address:
Password:
Remember me
Forgot your password?
No problem. Click here to have it e-mailed to you.

Not Registered Yet?

Register now for FREE. Registration only takes a few minutes to complete. Register now »