Information Wants to Be Free. What About Killer Information?

By Malcolm MacIver | December 27, 2011 1:52 pm

Malcolm MacIver is a bioengineer at Northwestern University who studies the neural and biomechanical basis of animal intelligence. He also consults for sci-fi films (e.g., Tron Legacy), and was the science advisor for the TV show Caprica

A few years ago, the world was aflame with fears about the virulent H5N1 avian flu, which infected several hundred people around the world and killed about 300 of them. The virus never acquired the ability to move between people, so it never became the pandemic we feared it might be. But recently virologists have discovered a way to mutate the bird flu virus that makes it more easily transmitted. The results were about to be published in Science and Nature when the U.S. government requested that the scientists and the journal withhold details of the method to make the virus. The journals have agreed to this request. Because the information being withheld is useful to many other scientists, access to the redacted paragraphs will be provided to researchers who pass a vetting process currently being established.

As a scientist, the idea of having any scientific work withheld is one that does not sit well. But then, I work mostly on “basic science,” which is science-speak for “unlikely to matter to anyone in the foreseeable future.” But in one area of work, my lab is developing new propulsion techniques for high-agility underwater robots and sensors that use weak electric fields to “see” in complete darkness or muddy water. This work, like a lot of engineering research, has the potential to be used in machines that harm people. I reassure myself of the morality of my efforts by the length of the chain of causation from my lab to such a device, which doesn’t seem much shorter than the chain for colleagues making better steels or more powerful engines. But having ruminated about my possible involvement with an Empire of Dark Knowledge, here’s my two cents about how to balance the right of free speech and academic freedom with dangerous consequences.

Consider the following thought experiment: suppose there really is a Big Red Button to launch the nukes, one in the U.S., and one in Russia, each currently restricted to their respective heads of government. Launching the nukes will surely result in the devastation of humanity. I’m running for president, and as part of my techno-libertarian ideology, I believe that “technology wants to be free” and I decide to put my money where my slogan is by providing every household in the U.S. with their very own Big Red Button (any resemblance to a real presidential candidate is purely accidental).

If you think this is a good idea, the rest of this post is unlikely to be of interest. But, if you agree that this is an extraordinarily bad idea, then let’s continue.

Now, let’s not be so device-centric. Let’s imagine that instead of a Big Red Button, we have an idea whose implementation is equally fatal to the continuation of humanity. Once again, we should expend no less effort preventing this idea from spreading than we did for the household Big Red Buttons. Our efforts of containment might not work—there’s lots of ways that an idea can escape, from Wikileaks to disgruntled employees—but it would surely be immoral to intentionally publish this lethal idea so that anyone could exact destruction on a vast scale. All efforts to control it and prevent release should be made. If it is a scientific idea, the science should not be published, and society should consider whether continued funding such research is justifiable.

Although it goes against my instinct, as a scientist, to hide any scientific results, I think the preceding logic compels just that at times when scientists generate dangerous knowledge.

Critics of the move to censor the bird flu information say it has already been presented at conferences, and that censoring it will hold back progress on the very science we may need to prevent a future outbreak. I don’t find “the cat’s already out of the bag” argument convincing in this case, since presentation at a conference of specialists is far from putting the result into a paper that can be downloaded anywhere in the world. Carl Zimmer presents a better, though still arguable, case that publishing the entire sequence wouldn’t present an undue risk. But even if our containment of dangerous knowledge is really shoddy, stymying only the Homer Simpsons of the world, it still prevents a large number of Homer Simpsons from committing a “doh” heard around the world.

In respect to concerns about putting brakes on the progress of science, our efforts to contain dangerous information should be proportional to how damaging its release could be. If the idea is literally one which would enable anyone to easily end humanity, then the controls will be very strict. One can imagine the horrifying possibility of having to quarantine the people who have the dangerous information. Clearly, less is called for in this case, since it’s harder to use this information on viruses to do harm. The vetting process that is currently being developed for the bird flu methods will surely not be perfect, but if it makes it harder for malevolent actors to get the information, then it is working to some extent.

So far, so obvious. What may be less obvious is how we should treat the censored scientists, for whom I have great empathy. Given that we may be holding back the success of these scientists for the benefit of society, serious efforts should be made to compensate them in proportion to the harm we are causing their careers. The withholding might have little effect: other bona fide virologists may easily pass the vetting process, and may be able to replicate and learn the methods in the new paper. On the other hand, the effects might be devastating. It usually takes many years to reach a result of the importance that theirs appears to be. It is possible that many exciting avenues building on this result will also have to not be pursued on threat of being similarly censored. They may lose out on a great deal of prestige and funding that would otherwise be theirs.

One form of compensation would give them funding to continue their current work, even though subsequent results may also be selectively withheld from publication. This would be helpful because funding depends on the judgment of your peers, and the blackout may interfere with peers seeing the crucial results. Another form of compensation would be to enable these scientists to modify their research if they wish to. Yet, as any scientist can attest, it is very expensive to change research directions. When a scientist in the life sciences starts their career, they are given what is called a “startup package” of between $500,000 and $2,000,000 to establish their research program. Since we have decided to put fetters on the scientists involved, I would argue that a similar amount or more should be provided to those who change research direction to something less likely to be hidden behind a government cloak.

Image: Flu virion, courtesy of CDC

  • Bobby LaVesh

    It would be naive to assume that researching what it takes to make a killer virus more easily transmitted between people would NOT be censored.

    Those researching this would almost certainly have been aware that they would be censored should this turn out to be feasible. They almost certainly researched this knowing it was a chance- and made the choice to continue anyway.

    I’d be more concerned for those that ACCIDENTALLY discovered something dangerous.

    In this case- this particular issue has been quite well published in the media. Being censored has probably made the researchers MORE famous- not less. That may not be the case for future researchers who get censored- it will no longer be “shocking-and-novel” to people that scientists are being quieted.

  • http://engineer CubistHamster

    So here I was, thinking you had a reasonable, cogent argument (albeit one I disagree with vehemently) and then you went an used “breaks” where you surely meant “brakes.” Just a little typo, to be sure, but one that jarred me back to sense and reminded me that SCIENCE IS FUNDAMENTALLY ABOUT FREE AND OPEN SHARING OF INFORMATION.

    Ed: Not sure that that typo means anything significant about science, but it’s fixed now.

  • Alhazred

    Let us not kid ourselves. There is a clear trajectory in the history of ‘scientific technological civilization’ (really the whole history and prehistory of humanity). More knowledge and technical capability are available to more and more people over time. Clearly any reasonable extrapolation of this trend more than at most a few more decades in the future leads not to a ‘big red button in every house’ but the ability of every household to build its own nuclear arsenal.

    While this may not bear directly on the cast at hand with H5N1, and attempt to impede this process is like putting your finger in the dike. Short of the disassembly of modern technological civilization and permanent halting of human progress we WILL have to deal with a day when practically anyone will be capable of destruction on a mass scale. Equally clearly MANY people aren’t capable of handling that power responsibly.

    Now, honestly ask yourself the question that this inevitably begs. Can humanity survive, even in the medium term? Really? Objectively? Forget about who to censor, the topic is simply irrelevant, like “what color do I paint my burning living room?”

  • Iain

    I agree.
    Learning from the past. The US gov’t published most of how to make a nuke after WWII thinking the Soviets were much closer than they actually were. Look at all the trouble that caused.

  • scribbler

    The first time I contemplated the ability of man to destroy mankind, I was give years old. A commentator was explaining “MAD” quite well for I understood that we and the USSR had the ability to wipe out all life on Earth. My father was disturbed and I assured him that surely, no one was stupid enough to try and wipe out all life on Earth.

    So far I have been right… ;-)

    None of this is new. We have been altering microbes to make them lethal enough to wipe out all humans for quite some time now in various weapons programs by various government entities around the world. I personally don’t see what it is about this “new” discovery that makes it any different than the other programs of said States.

    As for “censorship”, the information isn’t being withheld, it is being made much more difficult to access and a process is being put in place to keep the information from falling into the hands of some nut job who might use it to destroy human life on a massive scale. A practice in place in all of the above mentioned programs and a worthy pursuit, in my opinion. The right to survive of the many outweigh the rights of the few to intellectual exercise.

    The researches in this case HAD to know the implications of their findings and the possible horrific consequences IF this were to be used by those seeking to unleash death on a massive scale. I applaud their decision to acquiesce and keep this as quiet as possible BUT I also think them quite irresponsible for the manner in which they placed such potentially devastating knowledge at such risk of discovery by potential terrorist and other nut jobs. To reward them would, I think, not only be immoral but would set a very bad precedent. I mean, what would be the difference between creating a lethal virus and blackmailing the world by threatening its release and simply creating one to become a party of the proposed program to “compensate” scientists for imagined losses of the world protecting itself from the dangers of widespread release of potentially devastating knowledge?

    Lastly, will mankind survive the day when every person has the ability to destroy ever other human with ease? No…

  • LarryW

    One premise used is fundamentally incorrect. The idea that everyone has the knowledge to …… That is just plain silly. It is also not necessary to the argument.

    About 95% of the American public is scientifically and mathematically incompetent, and that value would approach 100% for the knowledge necessary to carry out the proposed mutations on the H5N1. I certainly would be part of that 100%, as even might be the author of this article, given the description of his research.

    All that is needed is a handful of competent enablers to create a DIY kit with instructions. Do you really believe that the meth labs peppering the US are run by scientists and not high school dropouts who read only at a 3rd grade level?

    Given the information already made public of the success of these two labs in creating a highly contagious and deadly version of the H5N1 virus, I doubt that the proposed censorship will have the effect being promised.

  • stan

    If the flu killed half the population,there would still be too many people on earth.

  • Tim

    Hang on a minute…..haven’t you folk noticed something? Scientific information is being regularly suppressed/withheld every day of the week. It doesn’t matter whether a scientist attempts to publish information relating to enhancing the virulence of an influenza virus, tries to obtain the formula of some toxin recently isolated from a microorganism or the method by which larvae of freshwater eels (Anquilla) are raised to the glass eel stage of their life cycle. This information is often very dangerous, patentable and/or worth a great deal of money. In such cases, (dare I suggest most) those who pay the piper call the tune. The said tune is played in a soundproof room with only a very select audience.
    This whole scenario isn’t new, why all the fuss?

  • scribbler

    LarryW, you defeat your own argument. The fact that high school drop outs are savvy enough to distribute massive harm with limited information not only proves that such knowledge is dangerous but that there are millions who should not have access to it.

    Mutating viruses and other microbes is within the reach of more people than you can imagine. Second hand machines that once were unattainable by the “average Joe” are now sold on eBay. I’ve actually priced centrifuges and gas chromatographs and found them both to be within my very limited budget.

    Not to mention the ease with which lethal chemicals capable of mass destruction can be made and unleashed upon society. It is a Miracle this hasn’t already been done. In the near future, if we progress as we have been, not destroying ourselves by accident is going to become a valid concern, let alone some nutso doing it on purpose…

  • Vincent Racaniello

    You write ‘when scientists generate dangerous knowledge.’ How do you know that the H5N1 information is dangerous? Do you think that the fatality rate is 60%? All of the press and many virologists believe this to be true, but in fact there is evidence that a good fraction of rural Asian populations have been infected with H5 viruses with little or no consequence. The 60% fatality comes from WHO-defined H5 cases (a restrictive definition) which are the most serious ones that end up in hospital. The fatality rate for seasonal influenza is 0.1%, but if you look at those individuals infected with seasonal influenza that end up in hospital, 20% of them die. It’s fine to be careful about dangerous experiments, but first you have to be sure they are dangerous.

  • Nicholas Wouters

    I think the danger comes not from the people with the know how, but the ones with half of the know how, who know just enough to start something dangerous, but now enough to contain it once it gets out of hand..

    I remember coming across a recipe for peroxy acetone on the internet and thinking this could be fun ! but after a few experiments, a few holes in my ceiling and almost blowing myself up I realised that what I was doing was incredibly stupid and stopped.

    I am all for freedom of information, but sometimes a little information in the wrong hands can be VERY dangerous (google David Hahn)


Discover's Newsletter

Sign up to get the latest science news delivered weekly right to your inbox!

The Crux

A collection of bright and big ideas about timely and important science from a community of experts.

See More

Collapse bottom bar

Login to your Account

E-mail address:
Remember me
Forgot your password?
No problem. Click here to have it e-mailed to you.

Not Registered Yet?

Register now for FREE. Registration only takes a few minutes to complete. Register now »