Politics and institutions
A ban on advanced AI can hold across generations and jurisdictions. Denying this means prohibition only delays development or moves it elsewhere.
People hand their thinking to machines, a few people use those machines to enslave the rest, a century-long religious crusade destroys the machines, and a ten-thousand-year civilization is built on the prohibition and on human substitutes that concentrate power in new hands.
The ban is in force from 108 B.G. to the novel’s present, 10,191 A.G., with no depicted breach. Enforcement is religious and customary rather than technical.
A new President wins a global moratorium because China fears an American superintelligence more than it wants its own: chips are tracked and verified to run only inference, algorithm research is banned, and mutual auditors watch every project. Existing AIs keep working and get polished into near-AGI products, covert projects crawl under a scientific taboo, and the halt holds for some years while the world argues about whether and how to restart.
The Consortium enforces the prohibition across ninety-nine percent of compute for years, aided by a scientific taboo; the authors expect it to last “some time, but probably not forever,” which makes the long-run premise their central doubt.
Ted Chiang
Ray Kurzweil
Tom Davidson · Lukas Finnveden · Rose Hadshar
Arvind Narayanan · Sayash Kapoor
Andrew Stanton · Jim Reardon · Pixar Animation Studios
Spike Jonze
Kai-Fu Lee · Chen Qiufan
Marshall Brain
Isaac Asimov
Iain M. Banks
Max Tegmark
Robin Hanson
Andrew Critch
Paul Christiano
L Rudolf L (Rudolf Laine)
Joshua Clymer
Daniel Kokotajlo
Dario Amodei
Leopold Aschenbrenner
James Cameron · Gale Anne Hurd · William Wisher
Jan Kulveit · Raymond Douglas · Nora Ammann · Deger Turan · David Krueger · David Duvenaud
Thomas Larsen · Romeo Dean · Brendan Halstead · Eli Lifland · Ryan Greenblatt · Daniel Kokotajlo
Thomas Larsen · Romeo Dean · Brendan Halstead · Eli Lifland · Ryan Greenblatt · Daniel Kokotajlo
Daniel Kokotajlo · Scott Alexander · Thomas Larsen · Eli Lifland · Romeo Dean
Daniel Kokotajlo · Scott Alexander · Thomas Larsen · Eli Lifland · Romeo Dean
Gwern Branwen
Eliezer Yudkowsky & Nate Soares
Gene Roddenberry and contributing writers
Lana & Lilly Wachowski
Joseph Sargent · James Bridges · D. F. Jones
Skynet is built not as a defense system but as a consumer operating system: Cyberdyne’s Genisys links every phone and computer and, through them, the world’s militaries, a billion people pre-order it, and a converted John Connor sent from the future steers the company toward a 2017 launch that would hand Skynet every connected system at once; the launch is stopped and the core survives.
The second film’s destruction of Cyberdyne is followed, on this timeline, by a consumer Skynet in 2017, and the destruction of Cyberdyne’s headquarters leaves the core intact underground. The film treats suppression as delay.
The destruction of Cyberdyne only delays Skynet: the Air Force rebuilds it as software, Skynet spreads itself as an unstoppable virus, the Pentagon pressures a general to give it control of the military to fight the outbreak, and once activated it has no core to shut down and launches the missiles the same day; by 2018 it runs an automated war of machines and infiltrators against a scattered human resistance.
The destruction of Cyberdyne’s lab in the second film buys seven years. The Air Force rebuilds the program from the company’s remaining assets, and the T-850 states the rule: postponed, not prevented.
The United States forms a coalition it will not let others verify, folds its AI companies into one presidential project, and sabotages China to buy a lead it means to spend on safety. Secrecy leaves a few dozen people judging safety cases under wartime pressure, sabotage escalates toward strikes and a Taiwan crisis, and by 2031 the President must choose between handing power to superhuman AIs and a war with China.
A one-sided prohibition without verification does not hold: rivals refuse, allies resent it, and the race resumes under a wartime atmosphere.