Imagen-Quantum Mechanics Explained: Sean Carroll on Many-Worlds Interpretation and Emergent Reality

In this thought-provoking installment of the Lex Fridman Podcast (episode #47), theoretical physicist Sean Carroll returns for his second appearance to explore the fascinating and often puzzling world of quantum mechanics, with a particular focus on the many-worlds interpretation that he details in his book "Something Deeply Hidden."

Sean Carroll, a theoretical physicist at Caltech and the Santa Fe Institute specializing in quantum mechanics, thermodynamics, cosmology, and gravitation, brings his remarkable ability to explain complex concepts with clarity and enthusiasm. Much like Bob Ross teaching painting techniques with calm assurance, Carroll guides us through the strange landscape of quantum physics in a way that makes it accessible without sacrificing depth.

The Philosophical Foundations of Physics

From Newton to Quantum Mechanics

The conversation begins with a discussion of classical mechanics, developed by Isaac Newton. As Carroll explains, Newton's theories worked remarkably well for predicting physical phenomena, but Newton himself was troubled by certain implications of his own work, particularly the idea of "action at a distance" – how the Sun could exert gravitational influence on Earth across 93 million miles of empty space.

"Newton literally said, 'I leave that for future generations to think about because I don't know what the answer is,'" Carroll notes. Future generations did indeed address this problem, beginning with Pierre-Simon Laplace around 1800, who reframed Newtonian gravity as a field theory. Rather than the Sun directly "reaching out" to Earth, there exists a gravitational field between them that obeys Laplace's equation.

This concept of fields pervading space became a cornerstone of modern physics, eventually leading to Einstein's theory of general relativity, which added the constraint that gravitational effects propagate at the speed of light.

When Lex asks whether we can truly understand concepts like fields, Carroll offers a thoughtful perspective on human understanding:

"Our intuitions get trained. I have different intuitions now than I had when I was a baby. That's okay. Intuition is not necessarily intrinsic to who we are; we can train it a little bit."

The Gap Between Perception and Reality

The conversation turns to the gap between what our perception systems allow us to see and what reality actually is. Carroll points out that while we don't naturally perceive atoms or quantum phenomena, our cognitive abilities combined with our sensory experiences have allowed us to develop scientific understanding far beyond our immediate perceptions.

When asked about the most beautiful idea in physics, Carroll doesn't hesitate: "Conservation of momentum."

He explains that for Aristotle, the obvious observation was that objects in motion eventually stop moving unless continually pushed. It took over a thousand years for thinkers to realize that, absent friction and other resistances, objects would naturally continue moving at a constant velocity forever.

"That is the most beautiful idea in physics because it shifts us from a view of natures and teleology to a view of patterns in the world... You don't need natures and purposes and goals; you just need some patterns."

This represented a profound shift from understanding the world in terms of purposes to understanding it in terms of patterns that unfold according to mathematical laws.

Understanding Quantum Mechanics

The Basics of Quantum Mechanics

Carroll provides a clear explanation of quantum mechanics as the paradigm that replaced classical mechanics in the early 20th century:

"In classical mechanics, you have an object, it has a location, it has a velocity, and if you know the location and velocity of everything in the world, you can say what everything's going to do. Quantum mechanics has an aspect of it that is kind of on the same lines. There's something called a quantum state or the wave function, and there's an equation governing what the quantum state does."

But quantum mechanics differs dramatically in how it treats observation. In classical mechanics, observing a system doesn't fundamentally alter it – you simply see what's happening. In quantum mechanics, the textbook view suggests that observation causes the system to change its state dramatically.

"The electron in an atom is not orbiting in a circle; it's sort of spread out in a cloud. When you look at it, you don't see that cloud. When you look at it, it looks like a particle with a location. So it dramatically changes its state right away."

Atoms, Electrons, and Wave Functions

Carroll explains that atoms, once thought to be indivisible, are now understood as structures composed of a nucleus (made of protons and neutrons) surrounded by electrons. While the nucleus contains most of the atom's mass, it's the lighter electrons that are responsible for chemical properties and electrical conductivity.

The wave function, which Carroll humorously notes has "the dopiest name in the world for one of the most profound things in the universe," represents the quantum state of a system. For an electron, the wave function can be visualized as a number at every point in space, indicating the value of the electron's wave function at that point. Crucially, there's only one wave function for the entire universe, not separate wave functions for each particle.

Entanglement and Hilbert Space

Entanglement occurs when the quantum states of particles become correlated in such a way that the state of one particle cannot be described independently of the others, regardless of the distance between them.

"If you see one in one location, then you know the other one's going to be doing a certain thing. So that's a feature of quantum mechanics that is nowhere to be found in classical mechanics."

Carroll introduces Hilbert space as the mathematical framework in which quantum wave functions exist. Unlike the three-dimensional space of our experience, Hilbert space is an abstract mathematical space with potentially infinite dimensions. Each dimension in Hilbert space corresponds to a piece of information needed to describe a quantum system.

The Many-Worlds Interpretation

What Are the Many Worlds?

Carroll advocates for the many-worlds interpretation of quantum mechanics, first proposed by Hugh Everett III in the 1950s:

"My favorite is the many-worlds interpretation, which says two things: number one, you the observer are just a quantum system like anything else... number two, when you think you're measuring something or observing something, what's really happening is you're becoming entangled with that thing."

In this interpretation, when you observe an electron that exists in multiple possible states, what happens is not that the electron "collapses" to one state, but rather that you become entangled with all possible states of the electron. The wave function now includes branches where "the electron was here and you think you saw it there, the electron was there and you think you saw it there," and so on.

These branches, once formed, no longer interact with each other – they become, effectively, separate worlds.

"Everett did therapy more than anything else. He said, 'It's okay, you don't need all these extra rules. All you need to do is believe the Schrödinger equation.' The cost is there's a whole bunch of extra worlds out there."

How Many Worlds Are There?

When asked how many worlds exist in this interpretation, Carroll explains that it's either an infinite number or a very large finite number. If our universe continues to accelerate in its expansion, creating a horizon beyond which we can never observe, theoretical arguments suggest that within our observable universe, there would be a finite number of possible quantum states – specifically, 10 to the power of 10 to the power of 122.

"Just to compare, the age of the universe is something like 10 to the 14 seconds... the number of particles in the universe is 10 to the 88th. But the number of dimensions of Hilbert space is 10 to the 10 to the 122, so that's just a crazy thing."

Common Concerns About Many-Worlds

Carroll addresses several common concerns about many-worlds, including whether it violates conservation of energy:

"There's zero question about whether or not many-worlds violates conservation of energy. It does not. I say this definitively because there are other questions I think there's answers to, but they're legitimate questions... This conservation of energy question, we know the answer to."

He explains that it's better to think of the universe as "splitting" rather than creating new copies, comparing it to representing a vector in mathematics as the sum of component vectors – the total "amount" of universe remains the same, just divided differently.

When asked why many-worlds remains controversial despite being, in Carroll's view, the cleanest interpretation of quantum mechanics, he explains that while the theory itself is simple, mapping it onto our everyday experience is challenging:

"Many-worlds is the version of quantum mechanics where it is hardest to map the underlying formalism to reality. That's where the lack of simplicity comes in, not in the theory but in how we use the theory to map onto reality."

Alternative Interpretations of Quantum Mechanics

Carroll outlines the three major contenders in quantum interpretation:

1. Many-worlds: The wave function is all that exists and always obeys the Schrödinger equation.
2. Hidden variable theories: The wave function is real but there's something additional. The simplest version suggests there are actual particles being pushed around by the wave function.
3. Spontaneous collapse theories: The wave function sometimes follows the Schrödinger equation and sometimes spontaneously collapses.

A fourth category, which Carroll calls "epistemic interpretations," suggests the wave function is just a way of making predictions rather than representing reality itself.

Quantum Mechanics and Fundamental Reality

Space-Time as Emergent

One of the most profound implications of modern physics is that space and time themselves might be emergent rather than fundamental. Carroll explains that when trying to combine general relativity (which describes gravity as the curvature of space-time) with quantum mechanics, physicists have encountered persistent problems.

These difficulties suggest that the quantum theory of gravity might not be a field theory at all, but something with "weird non-local features built into it somehow." This has led some physicists to consider that space-time itself may be emergent from more fundamental quantum processes.

"The fundamental description of the world does not include the word 'space.' It'll be something like a vector in Hilbert space, and you have to say, well, why is there a good approximate description which involves three-dimensional space and stuff inside it?"

When Lex asks what Carroll believes is truly fundamental in our universe, he answers simply: "A wave function living in Hilbert space... It's a vector in a 10 to the 10 to the 122-dimensional vector space. It's a complex vector with unit norm. It evolves according to the Schrödinger equation."

Testing Our Theories

Carroll notes that there are ongoing experiments to test whether wave functions spontaneously collapse, which would rule out the many-worlds interpretation if confirmed. However, testing theories about emergent space-time is more challenging because "we don't even really have a safely written-down, respectable, honest theory yet."

Potential observable effects might include violations of fundamental principles like the consistency of the speed of light across different wavelengths, but these aren't definite predictions because the theories are still developing.

Beyond Physics: Consciousness and Human Nature

When asked if quantum mechanics can teach us about human nature or consciousness, Carroll is skeptical:

"Not really is the short answer. Minds are pretty classical. I don't think that phenomena like entanglement are crucial to how the human mind works or about consciousness."

Carroll identifies himself as a "thoroughgoing physicalist" who believes mental states and properties are emergent, just like space-time. The fact that we don't yet understand consciousness is "not at all surprising" given the complexity involved.

Reflections on Science Communication

The conversation concludes with reflections on Carroll's own podcast, "Mindscape." He shares that he often learns the most from conversations outside his expertise – discussions about jazz with Wynton Marsalis, wine with a Master Sommelier, or the mechanics of making blockbuster movies with director Scott Derrickson.

Carroll acknowledges that he's still learning to be a good interviewer, balancing when to offer his own perspective versus when to let his guests speak. He emphasizes that he only invites guests from whom the audience can learn something interesting, regardless of whether he agrees with them:

"I have zero interest in bringing on people for whom I don't have any intellectual respect... I will happily bring on people with whom I disagree, but only people from whom I think the audience can learn something interesting."

Key Points

1. Quantum mechanics fundamentally changed our understanding of reality by showing that particles exist as probability waves until observed, and that entanglement allows particles to be correlated across vast distances.
2. The many-worlds interpretation offers the mathematically simplest explanation of quantum phenomena, proposing that all possible outcomes of quantum measurements occur in separate, branching worlds.
3. Space and time themselves may be emergent properties rather than fundamental aspects of reality, with the underlying reality being a wave function evolving in an abstract mathematical space.
4. Conservation of momentum represents one of the most beautiful ideas in physics because it shifted our understanding from teleology (purposes) to patterns following mathematical laws.
5. The wave function of the universe may exist in a space with an almost inconceivably large number of dimensions (10^10^122), far exceeding the number of particles in the observable universe.
6. Our intuitions about reality can be trained and expanded beyond our direct sensory experiences, allowing us to understand phenomena we cannot directly perceive.
7. Different interpretations of quantum mechanics make the same predictions about observable phenomena, making it difficult to experimentally determine which is correct.

For the full conversation, watch the video here