Donald O. Hebb

Every modern neural network learns by changing the strength of connections between simple units. The idea that learning is connection strengthening, and that the rule for it is local to the two cells involved, was stated clearly for the first time by Donald Hebb in 1949. That single proposal, later called Hebbian learning, is the ancestor of the weight update at the heart of deep learning today.

Who he was

Hebb was born in Chester, Nova Scotia, in 1904. He came to psychology late, after a stint as a schoolteacher and a failed attempt at writing novels. He earned his PhD at Harvard in 1936 under Karl Lashley, then worked with the neurosurgeon Wilder Penfield at the Montreal Neurological Institute, studying what happened to patients after parts of their brains were removed. He was surprised by how little large lesions affected intelligence, which pushed him to think about memory as distributed across many cells rather than stored in one place.

After several years at the Yerkes primate laboratories, he joined McGill University in 1947 and built its psychology department into one of the most influential in the world. His students included Brenda Milner, whose work on the patient H.M. defined the modern study of memory. He served as president of the American Psychological Association in 1960.

The 1949 book

The Organization of Behavior: A Neuropsychological Theory set out to explain perception, learning, and thought in terms of neurons, at a time when mainstream psychology treated the brain as a black box. Its most cited passage is a postulate about how a synapse changes when two cells are active together:

When an axon of cell A is near enough to excite a cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic change takes place in one or both cells such that A's efficiency, as one of the cells firing B, is increased. Donald Hebb, The Organization of Behavior, 1949

The slogan "cells that fire together wire together" is a later paraphrase, popularized in the 1990s, but it captures the postulate well. Hebb went further than the synapse. He argued that co-active neurons form cell assemblies, groups that come to represent a perceived object or idea, and that sequences of assemblies, which he called phase sequences, are the physical basis of a train of thought.

Why it matters for AI

Written as an equation, Hebb's postulate becomes a learning rule that any computer can run. If two units have activities x and y, the connection between them changes in proportion to their product:

• It made learning computable. Before Hebb, neural network models such as McCulloch and Pitts (1943) had fixed wiring. Hebb supplied the missing piece: a mechanism by which experience changes the network. Rosenblatt's perceptron (1958) and everything after it are attempts to state a better rule for the same quantity, the connection weight.
• It is the basis of associative memory. Hopfield networks (1982) store patterns using exactly Hebb's outer product rule, and recall them from partial cues. This work was central to the revival of neural network research in the 1980s and to the 2024 Nobel Prize in Physics awarded to Hopfield and Hinton.
• It underpins unsupervised learning. Oja's rule (1982) stabilizes Hebbian learning and makes a neuron extract the principal component of its inputs. Kohonen's self-organizing maps and the BCM theory of synaptic plasticity are Hebbian at their core. Modern self-supervised and contrastive methods are often described in the same terms: strengthen what co-occurs, weaken what does not.
• Cell assemblies anticipated distributed representation. Hebb's claim that a concept is a pattern of activity across many cells, not a single labeled unit, is the position that connectionism adopted in the 1980s and that embeddings in today's models take for granted.
• Biology confirmed it. Long-term potentiation, first described in 1973, and spike-timing dependent plasticity, measured in the late 1990s, showed that real synapses follow rules very close to Hebb's postulate. This keeps Hebbian learning a live topic in neuromorphic hardware and in efforts to make learning more local and energy efficient than backpropagation.

Timeline

1. 1904Born in Chester, Nova Scotia.
2. 1936PhD, Harvard, supervised by Karl Lashley.
3. 1937Joins Wilder Penfield at the Montreal Neurological Institute to study patients after brain surgery.
4. 1947Professor at McGill University; later chair of psychology and chancellor.
5. 1949Publishes The Organization of Behavior.
6. 1960President of the American Psychological Association.
7. 1982Hopfield networks and Oja's rule bring Hebbian learning to the center of neural network research.
8. 1985Dies in Chester, Nova Scotia.