Guest Post: Caleb Scharf on the Shadow Biosphere

By Sean Carroll | May 6, 2010 9:50 am

Caleb ScharfWe’ve been talking about life quite a bit here recently at Cosmic Variance, and it’s always fun to talk about areas in which one has absolutely no professional expertise. But it’s also fun to bring in experts, which is why we’re happy to welcome Caleb Scharf as a guest blogger. Caleb is Director of Astrobiology at Columbia University, author of a textbook on the subject, an recently jumped into blogging. In this post he reminds us that we’re still learning a lot about the forms of life right here on Earth — knowledge that will be invaluable as we search for it elsewhere.


It’s a real privilege to be able to write a guest blog for Cosmic Variance and to take a little side trip from my regular postings to Life, Unbounded – the science of origins.

The modern search for life in the universe encompasses everything from exoplanets and astrochemistry to geophysics and paleontology. Underlying and motivating the investigations in these fields – collectively labeled astrobiology – there are some fundamental assumptions, but do they make sense?

In recent weeks one might be forgiven for thinking that a shadowy biosphere surrounds us, aliens are poised to dismantle civilization, and that time traveling species are flitting in and out of view like barflies on a Saturday night. It’s a little disconcerting, does the Kool Aid have something special in it this Spring?

Unfortunately I think that all of these headline grabbing items miss the real story of what life is, here on Earth and potentially further afield. The idea of ‘shadow biospheres’ or multiple origins of terrestrial life sounds intriguing, and certainly helps bring focus to the fact that we can be very blinkered in our outlook. It also steers attention away from a more interesting and demonstrably real point.

microbes In the past couple of decades we have found a shadow biosphere, except that far from lurking in the cracks it turns out to be the biggest, most critical, biosphere on the planet. An astonishing 99.9% of life on Earth cannot be coerced to grow in a lab, and so we have overlooked it. Microbial life – single-celled bacteria and our ancient cousins the Archaea – is not just the stuff under your fingernails, it is what makes multi-cellular life like us function, and it helps govern the grand chemical cycles of our planet, from the continents to the oceans to the atmosphere. Such organisms have, over three to four billion years, evolved into an eye popping array of microscopic machines, the ultimate nano-bots. They can extract energy and raw materials from, it seems, almost any environment. A particularly good example is Desulforudis audaxviator – discovered 2.8 km down in a South African gold mine in a pocket of isolated water. Little audaxviator lives all alone when the vast majority of microbial life is utterly reliant on colonial symbiosis. It earns a living by mopping up the molecular detritus left after radioactive decay in the uranium rich rocks dissociates water and bicarbonates. That’s a very, very neat trick.

Twenty or thirty years ago we barely understood that such life existed on this planet. Now we are beginning to see that the longevity of our biosphere owes itself to precisely this crowd of ‘shadowy’ organisms. A truly wonderful paper was published a couple of years ago in which Falkowski, Fenchel and Delong laid out the big picture for life on Earth. In essence, they argue that single-celled microbial life is the manifestation of an even deeper truth; the core planetary gene set. This is the set of recipes for metabolism, or how to harvest a planet for energy, and we all rely on them. The result of billions of years of natural selection, these genes are widely dispersed across the microbial biosphere. This is true to such an extent that should 99% of life be wiped out by an asteroid collision, supervolcano, or dirty telephone receiver, the information for the molecular machinery that drives all organisms will be safely preserved in the surviving 1%. The living world does not end, it just reboots. Because of this, carbon-based life is a far more robust phenomenon than we could have ever imagined. It is the ultimate, Google-like, cloud computer.

Still though, isn’t this also a blinkered view of what might constitute life? Well, sure, but there’s another fact to consider. When we look out into the universe we find that the chemistry of our life – carbon based molecular structures – is not just occasional, it’s ubiquitous. Carbon is a fabulous player; simple molecules, rings, chains, polymers, sheets, crystals, and great clumps of sooty particles abound. Some is produced directly from the huge outflows of cooling gas from old stars, much forms in the thick nebulae and proto-stellar cocoons that eventually give rise to planets. Thousands of recognizable organic molecules, including amino acids, are found in the treacly mix of some meteorites – the remains of our own ancient solar system. This is a chemical bonanza that must have played a role in setting the stage on the young planet Earth. If this is blinkered then stick a blindfold on me.

So life on Earth is tough and tenacious, and the building blocks are everywhere. Is this enough reason to think that a similar blueprint exists in other places across the universe? Well, it’s definitely motivation to go looking, and to go looking for the kind of exotica that we already know, rather than inventing new ones. Is this reason enough to think that ‘intelligent’ life exists somewhere else? That’s a tough call. Life on Earth did remarkably well for the past 3.5 billion years without us around, I don’t think there is anything that indicates we are more than an evolutionary oddity (albeit an incredible one). It’s a big universe though, with plenty of room for oddities, even if they turn out to be extremely familiar.

CATEGORIZED UNDER: Guest Post, Science
  • Mantis

    Personally I don’t see the need for astrobiology and would love to read a post explaining why such a discipline is warranted and what astrobiologists have achieved so far.

    Caleb: “A truly wonderful paper was published a couple of years ago in which Falkowski, Fenchel and Delong laid out the big picture for life on Earth. In essence, they argue that single-celled microbial life is the manifestation of an even deeper truth; the core planetary gene set. This is the set of recipes for metabolism, or how to harvest a planet for energy, and we all rely on them.”

    What exactly is so deep about this truth? It seems like a pretty straightforward consequence of the theory of evolution and abundance of horizontal gene transfer.

    And why call it “the core planetary gene set”? To make it sound more astrobiologish? The adjective planetary makes little sense, we don’t know for example lunar or comet gene sets, yes, the genes in question are specific to Earth but they are not specific to planets in general.
    The “core” also makes no sense, what core? There are plenty of alternative routes to generate energy, there is no core set of such genes present in every organism.
    Finally we should distinguish the gene set in question from many others not involved in harvesting energy like those who carry out anabolism or replication.
    Taking all this into account a much better description would be “the metabolic gene set.”

  • caleb

    Let me respond to a couple of those points. The term ‘core planetary gene set’ is the one used in the paper by Falkowski et al, published in Science in 2008, in this case it’s really just trying to emphasize that there are a surprisingly small number of known genes (about 1500) that seem to code for all the molecular machinery needed to perform metabolism (which encompasses the ten or so known metabolic pathways – photosynthesis, methanogenesis, o2 respiration, sulphate reduction etc.). So, ‘core’ means an essentially irreducible set, take any of these away and life and the planet changes significantly if not disastrously. You’re right that ‘metabolic gene set’ would be more specific sounding, but again, the point was I think to emphasize how critical these genes are. I don’t think anyone is trying to say that this is a universal truth applicable to all planets – but it sure is intriguing.

    What is deep about this (specific) truth? Well, it’s the result of extensive research by many, many people, it’s not an answer that just fell out of the sky, it’s a carefully considered, examined, partially tested, idea that no one had come to just on the basis of applying evolutionary theory and/or horizontal gene transfer. In retrospect I completely agree that it makes a lot of sense…most of the best scientific ideas do, with hindsight.

    As for poor old (new ?) astrobiology. Well, it may surprise you that I actually agree that ‘astrobiology’ is not so much a unique discipline, but that it’s a convenient way of labeling an array of research done in different classical disciplines, all of which nonetheless is tackling the common questions of ‘life in the universe’. For example, I do not call myself an astrobiologist – I’m an astrophysicist by trade, who happens to work on exoplanets and the questions relating to how we can go look for life beyond the Earth. One of the reasons we use the term ‘astrobiology’ is because historically it has been hard to get people in different classical scientific disciplines to talk to each other, beyond casual bar room fare. Consider it an experiment to attach this label in the hope that people feel less embarrassed to admit that they’re interested in ‘aliens’. The fact is though that within the classical disciplines (and yes, under the label of astrobiology) there have been amazing advances in understanding extremophilic organisms on Earth – often motivated *precisely* because of what might be happening on Mars or Europa or elsewhere. In astronomy the big drive is to find the terrestrial type planets (NASA’s Kepler mission is designed to provide a statistical measurement of how many such planets exist) – why ? Because the idea of finding a planet like the Earth, that could harbor life is so compelling and interesting. Does this need to be called astrobiology ? No, but that is what it is….

  • Brian137

    Hi Caleb,
    Thank you for the post.
    We see that life on Earth exists under extremely diverse conditions, extracting energy through a number of different mechanisms. That fact encourages us to speculate that carbon-based life might be rather widespread throughout the universe. But another factor might be the ease or difficulty of life beginning ex nihilo where no life existed previously.

    So the question which interests me most right now is whether life on Earth began only once or whether it had at least two completely independent beginnings. If all our diverse biospheres evolved from the same beginning, then life may be a very rare occurrence despite its ability to evolve and adapt once initiated. I hope I have made myself clear, and I would welcome input from Caleb or anyone else.

  • caleb

    In response to Brian137. This is an excellent question and gets to the heart of a longstanding issue. It’s common when people assemble ‘tree of life’ type diagrams (typically these days based on phylogenetics – the similarity of one or more key bits of genetic material shared by all organisms) to show the base roots or trunk of the tree all converging to one point in the genetic past. This is total conjecture, we don’t actually know whether the great domains of bacteria, archaea and eukarya (to which we belong) all have a single common ancestor, right now there isn’t enough information to join those final dots. By the same token, we don’t know whether ‘life’ arose more than once – perhaps on different parts of a young Earth, or in multiple wide spread molecular ‘events’ – indeed we don’t actually know how, when, where ‘life’ got going at all. It does seem likely that before the kinds of organisms we recognize today (and clearly we’re just beginning to scrape the surface of what’s out there) there may have been a so-called RNA-world, without the critical, but more inflexible storage mechanism of DNA. RNA molecules are versatile, and as well as carrying information they can perform the duties of enzymes and so on. It’s been argued that a whole proto-life system could run on RNA. Such a system would, I imagine, offer many possibilities for separate ‘upgrade’ events (wow, I seem to really like computer metaphors…).

    So, where am I going with this ? I think it’s all going to hinge on exactly how rich a chemical mix ends up on the surface (or in the subsurface) of a potential harbor like a terrestrial planet (other environments could work too). Rich and complex enough and then at the very earliest stages of self-organizing behavior I could imagine multiple styles of life arising, ultimately ending up in layers of symbiotic and endosymbiotic (fully merged) relationships – to the point where it’s impossible to disentangle the ancestral patterns. I’m hedging my bets – I’d say that very early on then independent beginnings might indeed have occurred, but that these are likely to merge and erase evidence for that history.

    Could a separate start to life have occurred later on – after recognizable things like bacteria were already around ? Maybe, but I’d think it’d still have the same RNA mix, or something similar, at its root, I also think it would have a hard time finding a free niche – unless it got very specialized – *or* quickly figured out how to join in the biochemical party with everyone else.

    I agree that the details have big implications for whether life is a common or rare phenomenon in the universe.

  • Mantis

    Thank you for clarifications.

    Astrobiology as you describe it – as a label for (mostly) “traditionally” trained scientists whose work has bearing on the question of life beyond Earth – makes much more sense to me then a completely separate discipline.

  • spyder

    Thank you hugely for sharing a few basic insights into your work and that of others like you. Now i must add yet another daily blog to my bookmarks and look forward to more detailed accounts.

  • Andy Fleming

    Thank you Caleb for sharing this with us… a mind-blowing post.

    My semi-informed speculation is that microbial life is definitely out there and common in the universe. The building blocks of life are found all over, and the physical laws that governed the evolution of life on this planet appear to be the same throughout the cosmos.

    It’s a gargantuan visible universe… which may be just an infinitesimally small part of something much larger, or even infinite. In a Cosmos so large and so old, perhaps infinitely old – everything that can happen has and will happen… probably a gazillion times. In an infinitely large universe, even an infinite number of times!

    The speculation that there are other intelligent species on other planets is the ultimate application of the Copernican Principle/Principle of Mediocrity. As a layperson, my knowledge is far from complete, but that we are no alone makes sense. It also makes sense however, bearing in mind interstellar distances, that it may be many centuries, if at all before we obtain conclusive proof of ETI. The mean distances between civilisations may be huge!

  • Low Math, Meekly Interacting

    Bit belated, but I loved the post, and am very happy to be made aware of the Life Unbounded blog.

    Thanks to Caleb, and Sean for hosting!

  • caleb

    Just wanted to register my appreciation of the comments – glad that this stimulates – and happy to have new audiences !

  • Andy Fleming

    A pleasure on my part caleb… it’s a privilege to be a part of the discussion and be party to have access to these blogs. Thank goodness for the ‘net!

  • wds

    I was wondering where the 99.9% figure comes from. I know it’s said to be high, but I somehow had gotten it into my head it was more like 60-70%. Any references for this number?

  • Caleb

    Ah, good call. Well, to be honest I’m quoting the 99.9% from what I’ve heard a couple of microbiologists state, so I don’t have a direct reference. Obviously this number is going to be open to some variation – since we almost certainly don’t yet know exactly how many distinct species there are on the planet, and it if you’re making this determination in the lab it will depend on exactly where the samples are coming from.

  • RBH

    Caleb wrote

    … we don’t actually know whether the great domains of bacteria, archaea and eukarya (to which we belong) all have a single common ancestor, right now there isn’t enough information to join those final dots.

    Time (and research) marches on: we know more about that this week than we did last week when that sentence was written. See Doug Theobald’s new paper in Nature. Doug will be blogging about it on Panda’s Thumb very soon, and Nick Matzke already has a summary up there.

  • Brian137

    Fascinating. Caleb’s post #4 above was a much appreciated response to my post #3 above, in which I asked whether life on Earth had begun only once, or whether it might have had several completely independent beginnings. Too cheap to pay Nature to read the whole article, I found this synopsis and commentary:

    But according to Theobold, his findings do not preclude the possibility of multiple independent starts to life on Earth.

    From the link:
    “Let’s say life originated independently multiple times, which UCA allows is possible,” said Theobald.

  • RBH

    Sure it’s possible that life originated independently many times, but the Theobald’s data indicate that only one of those occurrences has descendants now.

    I’m told Theobald will be blogging on his paper at Panda’s Thumb one of these days real soon now.

  • caleb

    Right, the new analysis cannot say whether there were other ‘grandparents’ whose genetic lines are now simply gone (as far as we know), nor can it say when those lines – if any – faded out. It also makes me wonder whether we’re looking at the consequences of an extinction event or other type of evolutionary bottleneck, where only a single species made it through. I put a blog entry on Life, Unbounded about the result. It is also worth emphasizing that the analysis is *statistical*, and based on a number of assumptions.

  • Brian Too

    I believe this approach may have legs. If genes really are selfish (Dawkins), they may have ancient survival skills that were severely tested in the earliest days of their existence.

    Do we have any information, or even informed speculation, on whether life dates back to the Late Heavy Bombardment? Or does the LHB itself fall into the category of informed speculation rather than established fact? The reason I ask is that I had understood the Moon’s formation to be known to stem from an impact event. Yet lately I hear that this is possible but not proven.

    Another thought: What about the notion that early life was RNA based? Would the existence of a core planetary gene set not be written in RNA, more than in DNA, if the RNA emergence theory was true?

    E.O. Wilson wrote about the blossoming of knowledge that occurs when 2 previously separate scientific disciplines meet, in Consilience. Astrobiology sounds like an example of such a meeting.

  • Caleb

    As far as I’m aware there is no real evidence for life at the time of the LHB – although stromatolite remains seem to date back to about 3.8 Gyr ago, ‘just’ after the LHB (which ran from about 4.1 to 3.8 Gyr ago). The difficulty of finding such old, preserved, rocks undoubtedly puts huge uncertainty on things.

    The LHB itself does seem very likely to have occurred. The lunar cratering record (with radioisotope rock dating of melted forms) strongly supports a peak in large impacts at this time. There is also an excellent explanation for the LHB from the study of how the orbital architecture of the solar system came to be the way it is. Specifically, at an early stage (about 4 Gyr ago), it seems that there was a reconfiguration of the outer planets (Neptune, Uranus) during which time Neptune in particular underwent strong gravitational interaction with all the remnant planetesimals/asteroids/dwarf planets in the outer solar system. As a consequence Neptune’s orbit expanded, and much of this remnant material of smaller bodies (part of which is now the Kuiper belt) was scattered inwards, with some ending up pelting the Earth-Moon system. This also, incidently, seems to explain the different internal structures of the Jovian moons Ganymede and Callisto – because of heating via large impacts.

    The Moon formed earlier, about 4.53 Gyr ago, and the model in which it is formed from the debris following the collision of the young Earth with a Mars-sized proto-planet is still very much the best bet – it neatly explains many things, including the relatively high angular momentum of the Earth-Moon system.

    The RNA world idea is also quite compelling, and you’re right, it might well have been the stage at which a core planetary gene set would have begun to ‘assemble’ – although presumably the metabolic processes in such a world might be rather different ?


Discover's Newsletter

Sign up to get the latest science news delivered weekly right to your inbox!

Cosmic Variance

Random samplings from a universe of ideas.

About Sean Carroll

Sean Carroll is a Senior Research Associate in the Department of Physics at the California Institute of Technology. His research interests include theoretical aspects of cosmology, field theory, and gravitation. His most recent book is The Particle at the End of the Universe, about the Large Hadron Collider and the search for the Higgs boson. Here are some of his favorite blog posts, home page, and email: carroll [at] .


See More

Collapse bottom bar