Neuroscience and AI: Tomaso Poggio on How Brain Research Drives Machine Intelligence

The Journey From Physics to Intelligence

Tomaso Poggio's scientific journey began with a childhood fascination with physics and Einstein's theory of relativity. What captivated him about Einstein wasn't just the theory itself, but how Einstein reached his groundbreaking conclusions through thought experiments—using pure intellectual power to uncover profound truths about physical reality.

"Einstein was a hero to me because he was able to make a major contribution to physics with, simplifying a bit, just a Gedanken experiment," Poggio explains. "The fact that just with the force of his thinking, of his mind, he could guide to something so deep in terms of physical reality... it was something absolutely fascinating—the power of intelligence, the power of the mind."

Interestingly, Poggio notes that Einstein wasn't the top student among his peers. In fact, he was "the worst of the five" PhD students in his cohort at ETH Zurich and the only one who didn't secure an academic position after graduation. This led him to the Patent Office where his revolutionary ideas took shape.

"There is a lot to be said about trying to do the opposite or something quite different from what other people are doing," Poggio reflects. "That's actually true for the stock market—never buy when everybody's buying—and also true for science."

The Ultimate Scientific Problem

For Poggio, the problem of intelligence represents the greatest challenge in science—even greater than understanding the origin of life or the universe. His reasoning is profound: intelligence is not just another scientific problem, but the very tool we use to solve all other problems.

"The problem of human intelligence became a real focus of my science and my research because it's really asking not only a question about science but even about the very tool we are using to do science, which is our brain," he explains. "How does our brain work? From where does it come? What are its limitations? Can we make it better?"

As a teenager, Poggio realized that even if he had Einstein's brilliance, he couldn't hope to solve all the important scientific problems in his lifetime. But what if he could solve the problem of intelligence itself? That would potentially allow for the creation of intelligence "ten times better or faster than Einstein," which could then tackle all other scientific challenges.

The Brain-AI Connection: Necessity or Inspiration?

A central question in AI development is whether we need to understand the human brain to build truly intelligent machines. Poggio offers a nuanced perspective, noting that while humans achieved flight without fully replicating how birds fly, recent AI breakthroughs have indeed been inspired by neuroscience.

"The recent history of the progress in AI in the last, say, five years or ten years has been that the main breakthroughs really start from neuroscience," he explains.

Two prime examples:

1. Reinforcement learning, a core algorithm in AlphaGo (which defeated world Go champion Lee Sedol), has roots in the work of Pavlov and later neuroscientists.
2. Deep learning, essential to systems like autonomous vehicles, was initially inspired by the work of Torsten Wiesel and David Hubel at Harvard in the 1960s, who studied the hierarchical visual processing in the brain.

"My personal bet is that there is a good chance neuroscience will continue to play a big role, maybe not in all the future breakthroughs, but in some of them at least," Poggio says.

Neural Networks: From Simple to Sophisticated

While artificial neural networks were once considered too simplistic compared to biological ones, Poggio notes they're much closer to the brain's architecture than previous AI approaches like mathematical logic systems (LISP, Prolog). "You have networks of neurons, which is what the brain is about," he observes, even if artificial neurons are "caricatures" of biological ones.

A key difference remains: deep learning currently requires enormous amounts of labeled data, while children learn from remarkably few examples. "A child can learn—you tell a child 'this is a car'—you don't need to say like ImageNet 'this is a car, this is a car, this is not a car' one million times."

Poggio believes this gap is where future breakthroughs will emerge, though he's skeptical that current approaches like GANs (Generative Adversarial Networks) are the complete answer. "It's like they think they can get out more than they put in. There's no free lunch."

The Brain's Specialized Architecture

When exploring how the brain works, Poggio debunks the once-popular notion that the brain is "equipotential"—where any part could perform any function. Instead, the brain has specialized modules, though with remarkable plasticity.

He shares a fascinating example from Marge Livingstone's research at Harvard, who raised baby monkeys deprived of seeing faces during their first weeks of life. When examined later, these monkeys lacked the typical face-recognition capabilities in the brain areas that normally process faces.

"My guess is that what evolution does in this case," Poggio explains, "is that there is a plastic area which is predetermined to be imprinted very easily, but the command from the gene is not detailed circuitry for a face template... instead the command from the gene is something like: 'imprint/memorize what you see most often in the first two weeks of life, especially in connection with food.'"

Intriguingly, some of these monkeys developed sensitivity to blue gloves instead of faces—because technicians wearing blue gloves were what delivered their milk.

The Mysteries of Deep Learning

Poggio offers compelling insights into why deep neural networks work so effectively despite being overparameterized (having more parameters than data points). Contrary to conventional wisdom in statistics—which suggests having more data points than parameters—neural networks do the opposite.

"One nice side effect of having this overparameterization is that when you look for the minima of a loss function, like stochastic gradient descent is doing," Poggio explains, "I made some calculations based on some old basic theorem of algebra called Bezout's theorem... the bottom line is that there are probably more minima for a typical deep network than atoms in the universe."

This abundance of solutions means finding good ones becomes statistically more likely. It's similar to having a system of linear equations with more unknowns than equations—you're guaranteed infinite solutions.

The Human Visual System and Compositional Learning

Poggio's work on vision has led him to important insights about why deep neural networks perform so well. The key lies in what he calls "compositionality"—the ability to build complex representations from simpler parts in a hierarchical manner.

"When you are interpreting an image, classifying an image, you don't need to look at all pixels at once, but you can compute something from small groups of pixels, and then you can compute something on the output of this local computation," he explains. "That is similar to what you do when you read the sentence. You don't need to read the first and the last letter, but you can read syllables, combine them in words, combine the words in sentences."

This compositional structure helps deep networks avoid the "curse of dimensionality" that would otherwise make learning impossible with high-dimensional inputs like images.

Ethics and Consciousness

On the challenging questions of ethics and consciousness in AI, Poggio believes ethics is likely something that can be learned. He points to the emerging field of "neuroscience of ethics"—distinct from the ethics of neuroscience—where researchers like Rebecca Saxe have identified specific brain areas involved in ethical judgments. These areas can even be stimulated with magnetic fields to change ethical decisions.

Regarding consciousness, Poggio differs from some colleagues like Christof Koch (his first graduate student). While many AI researchers believe they don't need to understand or incorporate consciousness to build intelligent systems, Poggio suggests that in an extended Turing test, consciousness might be necessary.

The Next Breakthrough and Keys to Scientific Success

When asked about the next major AI breakthrough, Poggio believes it will likely again be inspired by neuroscience, though he can't predict exactly what form it will take. He's particularly excited about visual intelligence research—understanding how we perceive and comprehend the world around us, including recognizing other agents and navigating spaces.

From his experience mentoring successful figures like Demis Hassabis (DeepMind), Christof Koch (Allen Institute), and Amnon Shashua (Mobileye), Poggio distills what makes for success in science: "Curiosity and having fun, and I think it's important also having fun with other curious minds. It's the people you surround yourself with too."

As a research leader, he advocates a specific approach to new ideas: initial enthusiasm followed by thoughtful critical examination. "When somebody comes with a new idea in the group, unless it's really stupid, you are always enthusiastic at first—for a few minutes, for a few hours. Then you start asking critical questions, testing. If people are very critical from the beginning, that's not good. You have to give ideas a chance to grow."

Key Points:

1. The study of intelligence is arguably the most profound scientific problem because it involves understanding the very tool we use for scientific discovery.
2. Neuroscience has directly inspired major AI breakthroughs like deep learning and reinforcement learning, and will likely continue to influence future advances.
3. While artificial neural networks are simplified compared to biological ones, they share the fundamental architecture of interconnected units that process information hierarchically.
4. One of the biggest gaps between human and machine learning is our ability to learn from very few examples, while current AI systems require massive labeled datasets.
5. Neural networks succeed partly because their overparameterization creates an abundance of potential solutions, making good ones statistically more likely to be found.
6. The compositional structure of deep networks aligns with how real-world problems are naturally structured, allowing them to avoid the "curse of dimensionality."
7. Areas like ethics and consciousness, once considered purely philosophical, may eventually be understood through neuroscience and potentially incorporated into AI systems.

For the full conversation, watch the video: