No, but not because of some fetish for a “theoretical understanding”, but because such a magic oracle would be of little help for designing novel experiments/instruments/gadgets/whatnot.

You write “With all the talk about decoherence and expanding bubbles of decoherence due to thermal radiation, aren’t we forgetting the crucial point about the von Neumann/Dirac definitions of QM? At the instant of a ‘measurement’ (or observation, whatever that is) the quantum state instantaneously jumps to an Eigenstate – non-locally and super-luminally. Experiment after experiment have confirmed this uncomfortable fact. We immediately go from the nice linear, local progression of amplitudes into a non-linear, non-local realm and then back again, whereupon the linear evolution of states resumes. This isn’t ‘consistent mathematics’. It’s an inconsistent step in an algorithm that we follow that turns out to give correct answers, an internal inconsistency within the basic framework of QM.”

Your statements indicate some misunderstandings about decoherence and density matrices — there’s no more “inconsistent mathematics” here. The system is open during a measurement, and open systems evolve nonlinearly according to the appropriate Lindblad equation; you can show that this causes precisely the sort of hyper-fast nonlinear evolution that we expect from a measurement. You can even compute how long the “collapse” takes to happen.

Citing the “von Neumann/Dirac” definitions as evidence that quantum mechanics is inconsistent is like citing Aristotle as evidence that biology is inconsistent — the subject has changed over the intervening eons.

P.S. – Link to Malament paper: http://www.socsci.uci.edu/~dmalamen/bio/papers/InDefenseofDogma.pdf

We’ve been just “shutting up and calculating” for about 80 years now, just executing the algorithm without being able to explain how or why it works, even as the math was expanded and extrapolated to our current QM field theories. There is no deep “mathematical logic” behind it; it’s just a procedure we’ve learned to follow. So let’s not forget that this is still what we’re doing — just following the algorithm without understanding it. And keep trying to make the huge intellectual leap this understanding seems to require.

On a related note: see the papers by David Malament (1996) and Hans Halvorson & Rob Clifton, among others, showing that it is impossible to have a relativistic field theory of localizable “particles”. So, we should also be careful about what we ascribe to the “wave functions” that describe the events we describe as “particle” detections, since we don’t really know what those “particles” are, either.

Of course any proposed description of the physical world should be useful for predicting the results of experiments (otherwise it’s untestable), but that doesn’t mean it’s *just* a tool for predicting the results of experiments.

The point of scientific experimentation is to learn about the world, not just to learn what the result of a particular experiment will be. To illustrate this: If you had a magic oracle that could tell you what the result of any experiment would be, but you had no underlying theoretical understanding of why those experiments gave those results, you wouldn’t just declare science complete. (At least, I should hope not.)

If the wave function isn’t real, but is just a calculational tool, then it’s no better than an oracle. Sure, you know how the inputs were manipulated to get the outputs, but if it doesn’t describe the underlying elements of reality, then the question of why its predictions hold true is an open one.

Regarding electrons being an approximation, sure they may be, but an approximation to what? Likewise, if the wave function isn’t real, what is? You can’t tell me scientists don’t care what the underlying nature of reality is. Until somebody mentions quantum mechanics, everyone admits they care very much. As Wallace said, particle physicists want to know if there’s a Higgs boson, cosmologists want to know what dark energy is, etc. They don’t just want a systematic but inexplicable procedure for predicting what happens when you smash protons together at high speed, or point telescopes at distant galaxies, or perform whatever other relevant experiments there may be.

“Shut up and calculate” is a cop-out. It’s the equivalent of a paleontologist saying, “As long as we know that pretending dinosaurs existed and extrapolating from there will lead us to correct conclusions about the fossil record (or whatever else paleontologists study), then we don’t care if dinosaurs actually existed or not. That’s a question for *philosophers*!”

You can wonder if a concept that explains all observations is “real” or not, but at that point you’re debating definitions of words that have no impact, as far as I can tell. In addition, most physicists understand these things are just approximations, so the answer to the question “is an electron real” is 1) probably not, and 2) who cares, anyway?

There are a host of relevant issues I’m ignorant of, I’m sure. And I’m glad someone is working on the philosophy of QM – scientists are indeed generally content to just ignore such thorny problems. But your presenting a ridiculous straw man of instrumentalism doesn’t make me trust you…

You write “if a density operator is to be interpreted probabilistically then we can perfectly well consider probability distributions over non-orthogonal states, in which case the problem of indeterminacy recurs.”

That seems like a significant logical leap, and an unexamined assumption. Why can we perfectly well consider probability distributions over non-orthogonal states? Even classically, a probability distribution is only sensible when it’s defined over a set of mutually exclusive possibilities — it’s nonsense to say that you’re considering a probability distribution for a coin of 50%=heads, 50%=metal. Analogously, the inability to define a density matrix whose eigenvectors are spin-z and spin-x is just telling you that spin-z and spin-x are not mutually exclusive in quantum mechanics.

Indeed, suppose you forget this and try to define a classical probability distribution over non-orthogonal spin-z and spin-x states, respectively |up> and |right>. If you forget about density matrices and just try to demand that they have classical respective probabilities p and (1-p), then if you try to compute any expectation values by manually weighting the matrix elements by p and (1-p), you find that the answer is equivalent to tracing the corresponding observable against the density matrix ρ obtained by naively adding p|up><up| + (1-p)|left><left|. So all predictions and all observed experimental results are equivalent to assuming that this particular ρ is the density matrix of the system, but if we diagonalize ρ we see that its true ontic basis is something else entirely.

So quantum mechanics is telling you that certain probability distributions are as senseless as, like I said, 50%=heads, 50%=metal, because the possibilities aren't mutually exclusive. When you try to go ahead anyway, quantum mechanics calls you out for picking an invalid probability scheme and forces a different probability distribution on you — talk about a helpful, responsible theory!

Next you write "the density operators of microscopic systems (like components of singlet states) can’t in general be consistently treated as probabilistic mixtures of pure states, because we can do interference experiments to rule out that interpretation." That's only true when we enlarge our scope beyond the system whose density matrix we're considering, but then we're cheating because we're really supposed to use that larger system's density matrix.

In other words, say I have a couple of entangled particles. If I compute one of their density matrices, then that tells me about the ontology at that moment for that one particle. Its dynamics will be nonlinear, because the system is open to the interactions of the other particles, but if I write down the appropriate Lindblad equation, then I can in principle compute the particle's density matrix at all times, and then I can predict its ontology and all observations about it at all times.

But if I enlarge my scope to include the other particles, then the one particle's density matrix isn't enough, but it's obvious why: Now I'm asking about a different system — a larger system — and so I need to use that larger system's density matrix. The larger system's density matrix may look quite different, and have a different-looking ontology, but it's a different system, so that's okay — insisting that the ontologies be classically agreeable is just a naive classical assumption.

For macroscopic systems, that weirdness is much less likely — decoherence propagates outward at essentially the speed of light due to irreversible thermal radiation, so systems and their slightly-larger supersystems have a common ontology.

But in all situations, we can consistently say that the ontology of a specific system is determined by that system's — and only that system's — own density matrix, with epistemic probabilities dictated by the density matrix's eigenvalues. The weirdness that slightly enlarging the system makes the ontology look suddenly very different is what occurs more often for tiny systems than big ones, but it doesn't alter the axiomatic structure laid out in this paragraph.

You write that "the natural reading of the maths does seem to be that the reason the open-system equation applies is that on a larger scale we’ve still got closed-system dynamics." But we never have a truly closed system, at least in a universe that's open (in the Einstein sense) — there's always irreversible thermal radiation outward. By localizing questions of ontology — look at a particular system's density matrix to determine its ontology — we eliminate the need to worry about making this interpretation depend on the question of whether the whole universe is an open or closed system.

Indeed, because there aren't really any actually, perfectly closed systems in Nature, there are only Lindblad equations — the Schrodinger equation is always just an approximation. But that doesn't mean that a unique direction of time is automatically picked out for the universe — there's still the question of why all actual Lindblad equations for all large physical systems in our universe seem to line up, describing an increase in entropy in the same time direction. So the arrow of time question is not trivially resolved at all — it's just as open and interesting a question as when you assume that the Schrodinger equation governs everything.

I'm not a Bohmian, but I don't agree with your statement that Bohmian mechanics necessarily violates Lorentz invariance. If you take the wave-functional of a quantum field, Psi(phi(x)), and write down its "Schrodinger equation", then taking the polar decomposition of Psi yields a pair of Bohm dynamical equations as before, and it's all perfectly relativistically invariant. Now the degrees of freedom aren't particle positions, of course, but field values, but it all works fine, at least for bosons.

The trouble is fermions — a fermionic field doesn't have ordinary-number classical values, but Grassmann anticommuting values. I've never seen a convincing way around this problem.

Your comments are greatly appreciated.

I’m pretty sure Prof. Valentini could cite some motivations for pilot wave theory. Anyhow, given that de Broglie/Bohm reproduces so many results of QM is it truly well established that pilot wave theory is genuinely different mathematics, rather than a different formulation? Matrix mechanics, wave mechanics, path integrals are all equivalent. What is the theorem(s) that show pilot waves aren’t? Not to be a bug on a plate, but this proof would be interesting to know about.

As for the objection that the extension of pilot waves would violate Lorentz invariance, it seems that in one respect the division between microscopic and macroscopic could reasonably be interpreted as a transition between different inertial frames. In this sense, is it not possible that orthodox quantum physics also is not Lorentz invariant under all conditions? The whole discussion is about how quantum physics, with its superpostions and unidirectional time doesn’t seem to apply in the everyday world. It might be an odd way of thinking but reading the loss of Lorentz covariance being the reason uniquely quantum phenomena disappear on a macroscopic level could be useful, possibly?

Despite the enormous experimental evidence for QM and the consistency of the mathematics, quantum physics cannot model the things we see. This may not technically be a paradox but it certainly provokes these endless discussions. Certainly it is why everyday speech calls quantum physics paradoxical.

Many worlds seems to scoff at the conservation of energy. I tended to think that was in the math too. Where did it go, and how was its disappearance motivated? Perhaps the MWI means that the observable universe may be defined as those spacetime events that obey this symmetry and all the others are in principle unobservable. But where does that come from the math?

It’s not that I can really judge between pilot wave and many worlds. But it seems clear that quantum weirdness just does not get abolished in any framework. PIlot wave lumps it all into the pilot wave, while many worlds lumps it into the multiverse. The mathematics are consistent. But if we are going to incorporate quantum physics into our model of the universe, an interpretation, even if its weirdness requires years to get used to, needs to be consistent with the everyday world. Or its not a scientific explanation. Or so I think. Relativistic weirdness was gradually assimilated, just like fields or the distinction between momentum and KE. If we can figure out the right kind of quantum weirdness we can get used to it.

Matt @57: Thanks for an interesting response. Here’s an attempt to answer.

I was probably a bit quick regarding indeterminacy of the density operator as a probabilistic mixture. You’re quite right about degeneracy being a non-issue, of course. Having said that, if a density operator is to be interpreted probabilistically then we can perfectly well consider probability distributions over non-orthogonal states, in which case the problem of indeterminacy recurs.

Regarding the larger point, look at it this way: the density operators of microscopic systems (like components of singlet states) can’t in general be consistently treated as probabilistic mixtures of pure states, because we can do interference experiments to rule out that interpretation. The density operators of macroscopic systems, agreed, can be. But conceptually, it still seems that a physically-interpreted density operator (the micro sort) is a different kind of thing from a probabilistically-interpreted density operator. So how did the one turn into another? Or put another way, if the physical goings on are determined by the pure state, how did we move from a pure state at the micro level to a probabilistic mixture of pure states at the macro level? Appeal to decoherence (so goes the usual argument) won’t do here, because ultimately we can always include the environment in the process, in which case the dynamics is still unitary.

I think you’re rejecting that last sentence. Effectively (again: I think!) you’re wanting to draw a principled distinction between open and closed systems, so that when a system (like Earth) really is open, the evolution really should be taken as giving us a probabilistically-interpretable density operator.

Bell (and most philosophers) would reject that basically on the grounds that it’s imprecise. It relies on a transition from closed to open process that’s by it’s nature not exactly definable. If we really are seeing a change in the fundamental nature of dynamics – from deterministic to indeterministic – we’d better have a precise criterion for it. I suspect you’re not much moved by that objection.

Let me raise some slightly different-flavoured objections (none are intended to be decisive):

(1) it’s potentially going to cause trouble for quantum cosmology.
(2) it’s noteworthy that (at least in the examples I’m aware of) open-system quantum equations are fairly reliably derived by embedding the system in a larger environment (an oscillator or spin bath, say), applying unitary dynamics, and then tracing the environment back out again. (Often we’ll apply the unitary process only for some infinitesimal time, but still.) So the natural reading of the maths does seem to be that the reason the open-system equation applies is that on a larger scale we’ve still got closed-system dynamics.
(3) It gets in the way of any attempt to understand, rather than postulate, the direction of time. If all dynamics is Schrodinger dynamics, then the underlying dynamics are time-symmetric, and we can ask what additional information (e.g., a low-entropy boundary condition) breaks the symmetry. But the Lindblad equation (say) is explicitly time-irreversible.

Finally, a quick comment on your last. You say “none of this is any weirder than pilot waves or many worlds”. I actually regard those two possibilities as in very different categories. The pilot wave theory is a modification of the mathematical structure of quantum physics. I think that’s a very bad idea, not because it’s weird but because it’s unmotivated (and because it’s very unclear how to extend it to relativistic field theory, and very clear that any such extension would violate Lorentz covariance). The many-worlds theory, despite the name (cf Chris @25) doesn’t make any alterations to the mathematics; it just takes the “states are physical” assumption and applies it to the Universe as a whole.

For a pair of spin-1/2 particles, the spin-singlet state is a measure-zero phenomenon. More generally, density matrices with degenerate eigenvalues are always measure-zero in the space of all density matrices. If there’s even the slightest deviation, the degeneracy is broken and the problem goes away. Since measure-zero phenomena are essentially unrealizable in practical experiments — there is absolutely no way to get a perfect spin-singlet in any realistic experiment — there’s really no problem here.

One can show by direct calculation that if the experimental apparatus has many degrees of freedom, then even tiny degeneracy breaking in the subject system becomes highly robust. (This is a point that David Albert misses in an appendix to his book on interpretations of quantum mechanics.)

If accommodating measure-zero possibilities is nonetheless demanded, then the interpretation is still tenable: The density matrix doesn’t pick out any one preferred diagonalizing ontic basis, but we can just interpret that to mean that the system’s underlying ontic state can be anything in the degenerate eigenspace.

Your second point, that “we know that measurement outcomes on the whole system can’t be represented by classical correlations,” is less clear to me. I think you mean to say that the apparatus+subject composition system is still in a pure state after the experiment.

But in any realistic experiment, the environment rapidly decoheres the whole apparatus+subject composite system so that its correlations become classical. Then there’s some super-big system (including the whole Earth after a fraction of a second) that still isn’t classical with respect to the correlations, but again, as I’ve been saying, that’s not the subject system (the cat, or the electron), or the apparatus, or even the Earth.

Asserting that the super-system must have an ontology that agrees with the ontology of all its subsystems in a classical sense is an unexamined assumption — essentially a form of the fallacy of composition/division — but dropping it doesn’t conflict with any observations or experiments. All observers would agree that the subject system and the apparatus and even the Earth are now well-described by nontrivial density matrices consistent with the experiment, but that the super-system is not.

Certainly none of this is any weirder than pilot waves or many worlds.

You’ve been very gracious — would you mind explaining my misunderstanding here?

Thanks!

I believe the FM radio wave from the 93.9 FM WNYC radio station is macroscopic and thus “real” without any decoherence of its underlying photons. I believe this wave can still show interference phenomena.

The actual mathematical process by which we’re obtaining density operators from pure states is by letting system A get entangled with system B and then tracing out system B. If systems A and B are spin-half particles and we prepare them in a singlet state, the probabilistic interpretation of the density operators of the separate systems isn’t available because (a) it’s indeterminate what the mixture is supposed to be; (b) more importantly, we know that measurement outcomes on the whole system can’t be represented by classical correlations.

Now, of course, if system A is Schrodinger’s cat, and system B is its environment, the sort of experiments that show up this problem are (to put it mildly) technically impossible to perform. So we can get away with applying a probability reading there (modulo my worry about time asymmetry previously). But now we’re back to the issue of needing to interpret the state differently at macro- and micro-scales, only this time the problem’s at the level of density operators rather than pure states.

Aaron F@53: Matt Leifer’s argument is very interesting and elegant and I can’t properly do it justice here; the soundbite answer would be, “why think that a non-commutative generalisation of probability is still probability?” It certainly doesn’t seem to be probability if probability=relative frequency, or probability=Bayesian degree of confidence. (Matt also raises a nice philosophical problem for many-worlds: if the world isn’t really probabilistic, how come there are so many analogies between the mathematical structure of quantum theory, and probability theory. I’m going to pass on that question to avoid derailing the discussion.)

Mark@54: false dichotomy. Even papers in pure maths and theoretical physics advance their argument for the most part in natural language, albeit using plenty of words that don’t turn up in everyday usage; that’s because communication and reasoning take place in language and maths isn’t a language. (Try saying “natural language is a piss-poor alternative to mathematics” in mathematics.) However, much (most?) of what theoretical physicists use language to talk about is abstract mathematics. If the level of maths on this thread is low, it’s mostly because I (and I think others) are trying to keep the discussion accessible to a wide audience.