From Logic Gates to LLMs — a continuous technical schematic across five eras.
1950s — The birth of the artificial neuron
The first schematic of machine thought: a single weighted node deciding between 0 and 1. The Perceptron partitions space with a hyperplane — intelligence reduced to summation and a threshold.
1980s–90s — Backpropagation, RNNs & the vanishing gradient
Backpropagation made deep networks trainable, and Recurrent Neural Networks extended them to streams of text — a hidden state passed token by token through time. But the same depth that gave power also drained the learning signal: each backward step multiplied the gradient by a fraction, and fractions compounded into silence. Early layers — early words — were forgotten. The model lost long-term context.
2013 — Word embeddings & vector arithmetic
In 2013, Word2Vec collapsed meaning into geometry. Each word was trained into a dense vector — a coordinate in a ~300-dimensional semantic space — so that similar words landed near each other and relations became directions. Gender, tense, and capital-city offsets emerged as translatable arrows. Language was no longer a list of symbols; it was spatial math you could add and subtract: [King] − [Man] + [Woman] ≈ [Queen].
2017 — Transformers & self-attention
Sequence modeling is reframed as routing. Every token attends to every other token; context weights form a matrix of relationships. Recurrence is discarded in favor of parallel, weighted lookup.
Present — LLMs, RLHF, and the human feedback loop
Scale begets emergence. Tokens stream as probability distributions, shaped by human preference into helpful, aligned output. The loop closes: the machine now answers, and we grade the answer.