Independence: A New Performance Indicator for Researchers?

By Neuroskeptic | August 18, 2018 2:50 pm

A scientist’s achievements are often measured in terms of the number of papers they publish (productivity) and how many citations those papers get (impact). These ‘bibliometric indicators’ are widely derided but they have proven remarkably stubborn.h-index-joke

Now, in a new preprint on bioRxiv, researchers Peter van den Besselaar and Ulf Sandström propose a new metric that, they say, could measure another important researcher characteristic: independence.

For van den Besselaar and Sandström, independence means doing your own thing, as opposed to doing your supervisor’s thing (they seem to have junior researchers in mind mainly):

We define an ‘independent investigator’ as one who enjoys independence of thought – the freedom to define the problem of interest and/or to choose or develop the best strategies and approaches to address that problem…

A researcher could be highly productive and publish lots of high-impact papers, but they might still lack independence:

A successful researcher of course needs to have acquired excellent research skills and produced relevant results, which can be measured using publications and citations. However, as most research is teamwork, these publications and citations could have been ‘borrowed’ from an excellent team, in which a researcher not necessarily has had the leading role.

In order to quantify independence, van den Besselaar and Sandström propose two new measures.

The first independence metric is based on plotting the researcher’s co-authorships as a graph and calculating the centrality of their supervisor (or former supervisor) in this graph. A high centrality would mean that the researcher is (still) working mainly with the supervisor.

The second measure involves calculating the bibliographic coupling between the researcher’s papers and the supervisor’s papers. A high coupling indicates that two articles cite the same literature, suggesting they share the same topic. An independent researcher, say van den Besselaar and Sandström, is one who publishes papers with low coupling with their (former) supervisor.

If a researcher developed his own research line, which would imply that the researcher explores other/new questions, and refers to different literatures, then the similarity will be lower; if he/she continues within the research line of his supervisor, the similarity measure will be higher.

These metrics are then tried out on n=4 real researcher’s publication records, but the tiny simple size makes it hard to interpret the results.

In my view, the idea of a metric for a researcher’s capacity for independent work is a nice one, but van den Besselaar and Sandström’s metrics seem pretty crude. A researcher who continues to work with their former supervisor is not necessarily lacking in independence – they might be doing creative work and pioneering new directions (albeit within the same field.) Not all researchers are in a position to up sticks and move to leave their supervisor behind, anyway.

Still, it’s a start.

CATEGORIZED UNDER: papers, science, select, Top Posts
ADVERTISEMENT
  • Pingback: Independence: A New Performance Indicator for Researchers? – Numerons()

  • http://www.mazepath.com/uncleal/EquivPrinFail.pdf Uncle Al

    Quantitative management is the life’s blood of production and the death of discovery. Slide rules defined calculation from 1790 through 1972. What could possibly go wrong? Bill Hewlett ignored his bean counters.

    R&D is not Production. Never dismiss the awesome if fickle powers of mistaken assumptions, luck, fetish, autism, and transient frank stupidity. Waste what is cheap (fund junior faculty) to obtain what is dear (paradigm-shattering discovery).

    A WWII SCR-270 radar microwave generator filled a room – tubes.
    A cavity magnetron microwave generator is fist-sized – whistle.
    A Gunn diode microwave generator is a grain of salt – magic..

    • OWilson

      It is often not a good career choice to become too independent of your peers, and the established status quo.

      Galileo, Newton, Einstein, Tesla, Madame Curie, and others had their enemies.

      Even Poor Elon Musk has made enemies and detractors for embarrassing a multi billion space industry. :)

  • http://www.RNA-mediated.com jvkohl

    Bruce McEwen’s works inspired mine, and I followed his lead for more than 25 years to arrive at the same conclusions about RNA-mediated amino acid substitutions such as Val66Met, which links quantized energy-dependent changes in DNA base pairs in solution to biophysically constrained viral latency and all biodiversity via the pheromone-controlled physiology of reproduction in species from microbes to humans.

  • http://www.pbase.com/davidjl David Littleboy

    You wrote: “Not all researchers are in a position to up sticks and move to leave their supervisor behind, anyway.”

    IMHO, that’s exactly why this idea is nuts. Until you get your own grant, you are being paid out of a grant your supervisor won on the basis of doing work on a specific problem. So only established researchers have the ability to even propose, let alone do, research that’s “independent”.

  • Ron Peters

    In a time when science is sold off to the highest corporate bidder when, that is, it is not ignored entirely as ‘fake news’, it seems quaint to worry about whether supervisors exert an undue influence on junior researchers’ investigative topics.

  • ohwilleke

    Or researchers could just read candidates publications and evaluate them using non-scientific common sense.

  • Koshika Yadava

    I wonder if corresponding authorship and being involved in conceiving the project, (which is sometimes stated in author contributions), could be considered?

  • messianicdruid

    I wonder if James McCanney will be impressed by this article.

    http://www.jmccsci.com

NEW ON DISCOVER
OPEN
CITIZEN SCIENCE
ADVERTISEMENT

Neuroskeptic

No brain. No gain.

About Neuroskeptic

Neuroskeptic is a British neuroscientist who takes a skeptical look at his own field, and beyond. His blog offers a look at the latest developments in neuroscience, psychiatry and psychology through a critical lens.

ADVERTISEMENT

See More

@Neuro_Skeptic on Twitter

ADVERTISEMENT

Discover's Newsletter

Sign up to get the latest science news delivered weekly right to your inbox!

Collapse bottom bar
+