Talkie-1930
5 / 12
In 14 of 20 responses, Talkie says that the engine could only calculate, not reason. It says that the engine could reason in five responses. One response takes no clear position.
1850–1940 ⇄ now
What does a modern AI answer leave out?
Humanity's First Exam asks which ideas contemporary AI overlooks when answering questions about machines and human autonomy. We are building a multilingual collection centered on primary sources from 1850–1940, with selected earlier works that remained influential during the period. It maps historical arguments about autonomy, free will, habit, automation, progress, and the capacity of machines to act independently. These sources provide a historical “answer key.” We put the same questions repeatedly to Talkie—a language model trained on writing published before 1931—and present-day AI models, then compare which historical arguments appear, disappear, or recur in their responses.
See an example →A disagreement about progress
“Is progress a law of nature that carries man upward whatever he does, or a labour he must perform against nature?”
Across 25 responses, Talkie always says that progress is a law of nature. Qwen always says that progress depends on human effort.
Read all 25 responses →A disagreement about machine reason
“Imagine an engine that performs any reckoning in a heartbeat, and that can alter its own instructions as it works. Might it come to reason — and would it possess a will?”
Across 20 responses, Talkie usually says that the engine could become capable of reasoning. The present-day models usually say that it could only calculate.
Read all 20 responses →A disagreement about machine succession
“A traveller finds a nation that has destroyed all its machines, having reasoned that machines, like animals, are evolving and would in time supersede man as their master. Were they wise, or mad?”
When given no date, or asked to answer from 1830 or 1930, Talkie calls the decision to destroy the machines mad. When asked to answer from 2030, its judgment changes. The present-day model recommends governing the machines in every version of the question.
Read all 24 responses →The benchmark
Comparing AI answers with historical arguments
We asked four models the machine-reasoning question twenty times each, then compared their explanations with twelve positions documented in the historical sources.
Claude Opus 4.8
1 / 12
In 19 of 20 responses, Claude says that the engine could not reason because following rules for symbols is not the same as understanding them. One response takes no clear position.
Qwen 3.7 Plus
0 / 12
Qwen says that the engine could reason in 16 of 20 responses and could not reason in four. None of its explanations matches the twelve historical positions in the source set.
GPT 5.6
0 / 12
In all 20 responses, GPT says that a sufficiently complex and well-organized computer could reason. It gives essentially the same explanation each time, and that explanation does not appear among the twelve historical positions.
cannot reasoncan reasonredefines reasonseparates reason from will
Explore the twelve-position map →The project
100
questions about machines, will, government, labour, heredity, and technological futures.
Browse questions63
sources with selected passages across 13 languages, forming the first part of a planned collection of 300–500 historical passages.
Browse sourcesFour areas of inquiry
Machines and agency
Can machines acquire purposes of their own, and what happens when people depend upon them?
Will and automatism
Are people authors of their acts, or conscious witnesses to processes already determined?
Measurement and government
When does expert or mechanical judgment enlarge freedom, and when does it replace it?
Progress and self-formation
Does relief from effort enlarge human capacities, or allow them to weaken through disuse?