I am often surprised at the level of disdain and resentment with which string theory is viewed by non-string-theorists. I’m thinking not so much of people on the street, but of physicists, other scientists, and even other academics. As a physicist who is not personally identified as a string theorist, I get to hear all sorts of disparaging remarks about the field from experimental particle physicists, condensed matter physicists, astrophysicists, chemists, philosophers, and so on. I sometimes wonder whether most string theorists understand all the suspicion directed against them.
It shouldn’t be like this. String theory, with all of its difficulties, is by far the most promising route to one of the most long-lasting and ambitious goals of natural science: a complete understanding of the microscopic laws of nature. In particular, it is by far the most promising way to reconcile gravity and quantum mechanics, the most important unsolved problem in fundamental physics. At the moment, it’s a notably incomplete and frustrating theory, but not without genuinely astonishing successes to its credit.
The basic idea is incredibly simple: instead of imagining that elementary particles are really fundamentally pointlike, imagine that they are one-dimensional loops or line segments — strings. Now just take that idea and try to make it consistent with the rules of relativity and quantum mechanics. Once you set off down this road, you are are inevitably led to a remarkably rich structure: extra dimensions, gauge theories, supersymmetry, new extended objects, dualities, holography, and who knows what else. Most impressively of all, you are led to gravity: one of the modes of a vibrating string corresponds to a massless spin-two particle, whose properties turn out to be that of a graviton. It’s really this feature that separates string theory from any other route to quantum gravity. In other approaches, you generally start with some way of representing curved spacetime and try to quantize it, soon getting more or less stuck. In string theory, you just say the word “strings,” and gravity leaps out at you whether you like it or not.
So why wouldn’t anyone be happy about string theory? For one thing, we don’t understand the theory very well. It’s easy to say “replace particles with strings,” but quantum field theory isn’t really about “particles” — particles are just the observable momentum eigenstates in a perturbative regime, not the fundamental building blocks of the theory. At this point it’s a little unclear what the fundamental building blocks of string theory are; there are some reasonable proposals for complete non-perturbative definitions of the theory (matrix theory and AdS/CFT, for those in the know), but connecting these formulations to a more complete picture isn’t easy.
But most of the grumbles about string theory from other physicists aren’t about a complete non-perturbative definition of the theory — they are about the lack of connection to experiments. One often hears that string theory simply makes no predictions, but that’s clearly false. If you scatter two particles together, string theory unambiguously predicts that the cross-section should look stringy, not like that of fundamental point particles. [With caveats discussed in the comments.] The problem, of course, is that the difference between these two possibilities is only noticeable when the energy of the collision approaches the Planck scale (or really the string scale, likely to be similar) — fantastically far away from what we can actually reach in accelerators. So string theory makes predictions, it’s just that we are as yet unable to test them. In other words, string theory is either right or wrong, it’s our challenge to come up with clever ways to figure out which.
There is a matter of principle here that scientists, of all people, should understand. To wit, our current understanding of nature — based on classical general relativity and the quantum-mechanical Standard Model of particle physics — is simply incoherent. It just doesn’t make logical sense. It is very easy to ask questions to which we don’t know the answer: “What is the gravitational field of an electron?” For that matter, since the Sun is made of elementary particles, we can’t even sensibly talk about the Sun as simultaneously a source of gravity and as a source of light and heat. This is not acceptable. Our goal as scientists is to understand how the world works, and relying simultaneously on theories that are deeply incompatible with each other is nothing to be happy with. Even if it won’t help us make a better TV set or understand the mass of the proton, we need to have a coherent theory of quantum gravity.
Recently there has arisen another sense in which string theory purportedly makes no predictions, associated with the “landscape” of possible string vacuum states. Just as in quantum field theory, the observable spectrum of low-energy string excitations and their interactions (that is to say, particle physics) depends not only on the fundamental string physics, but on the specific vacuum state in which we find ourselves. String theory predicts more spatial dimensions than we directly observe, so one of the characteristics of our vacuum is the way in which the extra dimensions are hidden from our view. It now seems quite plausible that the number of possible ways for this to happen is enormous — perhaps 10500 or so. If true, this puts a damper on the hope that string theory would predict a unique vacuum state, and we could explain (for example) the ratio of the muon mass to the electron mass from first principles.
Well, too bad. It would have been great to make such predictions, but the inability to do so doesn’t render string theory non-scientific. The appropriate comparison for string theory is not to “the Standard Model of Particle Physics,” it’s to “quantum field theory.” Nobody complains that there are a huge number of possible quantum field theories, and we actually have to go out and measure the properties of actual particles rather than calculating them using pure thought. If string theory turns out to be the same way, that’s life.
My own view is that string theorists have been a victim of their own characteristically aggressive form of optimism. Not only, we are told, is string theory a consistent theory of quantum gravity, but it’s a theory of everything, gives us wonderful new insights into gauge theories, and possesses a mathematical beauty that is so compelling that the theory simply must be correct. These kinds of arguments just don’t carry that much weight with the non-converted. If I were in charge of the string theory public-relations machine, I would be emphasizing over and over again the basic feature that we’ve understood for a very long time: it’s the most promising way we know to quantize gravity. If there were multiple very successful ways to quantize gravity, it would be important to distinguish between them experimentally; but so long as the number of successful models is less than or equal to one, it makes perfect sense to make every effort to understand that model.
Which is not to say that we shouldn’t also pursue alternatives. I’m all in favor of supporting research on loop quantum gravity, dynamical triangulations, causal sets, and whatever else smart physicists might personally find promising. As long as we don’t know what the correct theory is, individuals need to use their own judgment about what clues to follow. String theory, starting as it usually does from talk about perturbative excitations propagating in a background spacetime, will not seem especially compelling to someone who thinks that background-independence is the most profound feature of gravity. It’s certainly good to support plucky Apples and Linuxes in the face of the Microsoft-esque dominance of the string theory approach; you simply can’t tell ahead of time when someone will hit on a brilliant new idea.
On the other hand, string theory has thus far been fantastically more fruitful than any other idea. When you get into string theory, one of the things that keeps you going is that you don’t get stuck — the rate of progress waxes and wanes, but the progress is very real. It didn’t have to be true that the five string theories studied in the 1980′s would turn out to all be part of one big theory, but they are. It didn’t have to work out that the entropy of a black hole calculated from semiclassical gravity ala Hawking would be equal to the entropy of a corresponding gas of strings and branes, but it is. It’s clues like these that keep the believers moving forward, hoping to understand both the inner workings of the theory and its ultimate connection with what we observe. We interested outsiders should be cheering them on.