There’s A Place For Heroic Gambles In Science

By Neuroskeptic | July 16, 2014 11:48 am

Over at Paul Knoepfler’s excellent Stem Cell Blog, commenter Robert Geller (@rjgeller) offers some remarkable data about the hiring of a disgraced scientist.

Geller queries why Haruko Obokata, the biologist at the center of the “STAP” stem cell scandal, was ever given her job. Obokata is a Research Unit Leader (RUL) at Japan’s national Riken Center for Developmental Biology (CDB). It was after being appointed to this prestigious post that she completed and published her discovery of “STAP cells” – supposedly a new kind of way of making stem cells. Her data turned out to be serious flawed and Obokata’s two papers on STAP were retracted in Nature earlier this month. But should she have been hired in the first place?

Geller compares Obokata’s CV against those of five unnamed “researchers in Japan in the same general field as Dr. Obokata” (and with similar ages), and also against a sixth individual, a biologist who was given the job of Research Unit Leader at the Riken CDB at the same time as Obokata. Here’s the data, with Obokata in red:

obokata_geller

Geller says (my emphasis) that

Dr. Obokata had both the lowest number of total citations and least impactful “hit” [i.e. single most-cited paper] of any of the seven researchers. This suggests that, in the absence of some specific non-quantitative reason for rejecting the ranking implied by the citation data, Dr. Obokata should not have been hired by Riken. This considers only researchers inside Japan and it seems highly likely that there would have also been more highly qualified candidates [for the RUL post] than Dr. Obokata from outside Japan…

In a nutshell, Obokata’s ‘metrics‘ are poor. Compared to her peers she has not published many highly-cited papers. So, Geller asks, why was she, and not someone better qualified, given the extremely prestigious Riken RUL post?

Various reports sketch out a plausible story of what happened here. The story is that Obokata was head-hunted on the personal initiative of the CDB’s management, who wanted Obokata on their staff, so that their institute could claim credit for STAP. Their hope, we’re told, was that STAP would allow Riken to outshine their rival, Nobel Laureate biologist Shinya Yamanaka and his induced pluripotent stem cells (iPSCs). Had STAP cells been real, they would have made iPSCs obsolete.

So, by this account, Obokata was hired, not on the strength of her past published work (as STAP was still unpublished at that stage), but on the strength of her current vision and her future potential. Rather than citations and metrics, she was judged by her ideas.

Which is… great. That’s how it should be. This kind of thing ought to happen more often. Metrics can’t measure everything, and there’s a place in science for heroic leaps into the unknown.

So I don’t think the hiring of Obokata should be criticized – not as such. It wasn’t itself a mistake, rather it was based on a mistake, namely the idea that Obokata and her STAP were about to revolutionize biology. This was an error, a scientific blunder on a grand scale. But given that mistaken theory, hiring Obokata was a brave move, a high-stakes scientific gamble. Had it paid off, whoever made the decision would have been seen as a visionary, and the status of the CDB would have been enormously enhanced.

It would be a shame if the hiring of Obokata comes to be seen as a case in point that scientific appointments should be based on metrics. Rather, the lesson here is that even the most attractive ideas require critical evaluation.

  • Paul Knoepfler

    Going for the gusto to tackle a huge question or pursue a cool idea in science is one thing (and I’m a big fan of that), but it seems that unhealthy personal & institutional ambition played major roles here too in STAP. I’m not so sure I’d characterize the hiring as “brave”, but more like reckless. Why? Not because of weak metrics, but rather because of the lack of depth of real, convincing data supporting STAP and the hirers didn’t do their homework to check on her previous papers including the adult pluripotent stem cell one that had major red flags. If you are going to take a heroic leap it doesn’t mean that you shouldn’t do your homework too, right? In fact, it becomes all the more important.

    • http://blogs.discovermagazine.com/neuroskeptic/ Neuroskeptic

      I agree – that’s what I meant by “even the most attractive ideas require critical evaluation”, which clearly didn’t happen in this case.

      Riken hiring Obokata was a huge mistake because the STAP idea was flawed. But they weren’t wrong to go chasing an idea.

      As to whether the ambition to “keep up with Yamanaka” was unhealthy, it certainly seems ridiculous looking back, but it’s easy to say that in retrospect. If it had succeeded, we wouldn’t be saying that.

  • http://blogs.discovermagazine.com Longmire

    This is garbage your article makes it sound like she forged all the data and that STAP stem cells cant be created by the method she detailed which is not the case. All she did was make 4 mistakes labeling photos and 1 about the lineage of the cells. She is currently recreating the experiments under video surveillance, wow some pretty powerful people have a lot to lose from this discovery it seems. If anyone really wanted to see what a sad state of affairs science is in they could look at how few of attempts have been made to recreate the experiment or the concept thereof, and If you actually read closely “scientist” aren’t saying they cant create STAP cells but are having “trouble” duplicating the experiment and other vague nonsense.

    • http://blogs.discovermagazine.com/neuroskeptic/ Neuroskeptic

      If you read the post you’d see that I didn’t say she forged all the data, and that some pretty powerful people had a lot to gain from the discovery – the very people who are now putting her under video surveillance because they don’t trust her.

      • http://blogs.discovermagazine.com Longmire

        Of course you didn’t accuse her of forgery because she didn’t forge the data but every article I have read on the subject infers she has with comments like “Had STAP cells been real, they would have made iPSCs obsolete.” I have read other articles where people were saying that the truth may never be known wow are we really that ignorant. The surveillance is more likely meant to intimidate her and cause others to tread softly lest funding dry up. I cant help but believe this will be a major scandal in the future so realize the same future judges us all.

  • http://www.onyourmind.ca liam crapper

    This is a real pet topic for me. The idea that hiring should be purely metric based is ludicrous, and I would contend that previous publication record has limited predictive value for someones ability to be a creative and productive researcher. Unfortunately it does predict their ability to draw funding, which is based on the same metrics.

    Scientists, it would seem particularly those in a position to make hiring decisions, are a conservative bunch. They know that this it’s risky to hire based on anything other than metrics, because when something like this happens, or even when a hire ends up not excelling, everyone will look back at their track record. It’s a way for the committee to shield themselves,but it leaves very talented researchers with the misfortune of working in a lab that doesn’t publish a lot of high impact work in the lurch.

  • Charles

    Metric- versus potential-based hiring depends critically on the position. For example, taking someone with lots of variance (huge upside and huge downside) makes perfect sense for a Jr. faculty member. But, say, giving tenure to such a person is dangerous. In the first case, if the individual doesn’t pan out you truncate (deny tenure to) the left tail and you are left with just the successful ones (of course, there is always self-censuring of the right tail as successful people leave). But taking risks on people when you can’t get rid of them easily is completely different.
    In her case this was a prestigious position, firing her I am sure would probably be difficult. Plus she was a manager, you cannot discount the importance of a good resume for such a person. She needs the respect of juniors. Hiring thinking that respect may come from future really successful future publications may put the reputation of the institution at risk.

    • http://blogs.discovermagazine.com/neuroskeptic/ Neuroskeptic

      That’s very true, but (my understanding is) that RUL is a junior position, albeit a prestigious one – and is a 5 year contract (iirc). So it’s not as if they committed themselves to Obokata forever, by giving her that post.

      • Charles

        Then I agree. Taking risks on someone should be good. This notion that institutions should not take risks regrettably extends to admitting students to college. There is almost an implicit promise if we accept you, we will graduate you. I think this is a regrettable outcome in a world of imperfect information.

  • Dario

    It’s clear to me they just looked at political/financlial potential, not really at her “ideas”. This things are happening more and more often everywhere.
    It’s good they didn’t hire through metrics but they used even worse reasons!
    Instead of defining the hiring as an “heroic gamble”, I would rather say it’s a Miserable Speculation. With a right and predictable tragic ending.

  • Sake lover

    Your article seem to be written based on the idea all of the flaws in STAP article was just by “Dr Obokata”.
    But I think it is a little early to judge that “all of responsibilities” should be imposed on her,while she indeed has to take responsibilities as a first author.

    Because-
    1)She was in charge of producing STAP cells in article part, and Dr Wakayama,one of the co-authors, was in charge of producing STAP -SC in the letter part.

    2)The mice genetic test,which indicated the fetal flaw to prove the existence of STAP, has flaw(again) in itself.
    After the result of mice genetic test was announced by Dr Wakayama and RIKEN ,all of co-authors,including Dr.Obokata, retracted the article and letter.
    Because the test indicated that the mice used for STAP-cells and the one used for STAP-SC were different.
    At the point,the data for proving the existence of STAP cells were vanished.
    However,few days later,Dr Wakayama and RIKEN admitted that there are flaws in “genetic test” itself and now RIKEN and Dr Wakayama are investigating the credibility of “genetic test”.

    3)Now, Dr Obokata is trying to verify the STAP existence in the lab with some surveillance cameras.
    After finishing her verification,it will be clear that all of her idea was true or false or partially true(false).

    I know most of researchers are so irritated about the fuss around STAP, but I would really like to ask you all to be patient a little more to judge STAP theory and herself.

    DON’T CRITICIZE HER BEFORE KNOWING THE TRUTH.

    Thank you.

    *P.S.
    1 It should be taken into consideration that she could not enter the lab for couple of month,because RIKEN committee’s permission for her to enter the lab was delayed.
    2 I am taking a supportive position on her,but this doesn’t mean support the conducts she did,such as falsification of images.

NEW ON DISCOVER
OPEN
CITIZEN SCIENCE
ADVERTISEMENT

Discover's Newsletter

Sign up to get the latest science news delivered weekly right to your inbox!

Neuroskeptic

No brain. No gain.

About Neuroskeptic

Neuroskeptic is a British neuroscientist who takes a skeptical look at his own field, and beyond. His blog offers a look at the latest developments in neuroscience, psychiatry and psychology through a critical lens.

ADVERTISEMENT

See More

@Neuro_Skeptic on Twitter

ADVERTISEMENT
Collapse bottom bar
+

Login to your Account

X
E-mail address:
Password:
Remember me
Forgot your password?
No problem. Click here to have it e-mailed to you.

Not Registered Yet?

Register now for FREE. Registration only takes a few minutes to complete. Register now »