Aligning individual AI systems with their users will not be sufficient to preserve humanity’s collective control.

The thesis: agent-specific interventions such as “aligning or shutting down this or that AI system or company” will not be enough to avert an agent-agnostic process, and the post quotes AI Impacts on aligned-but-myopic use destroying the future.

The closing section says the technology to control the AIs’ goals arrived in time, “but this alone does not let you control the future,” because what emerges is the solution to “what propagates fastest?”

The paper’s thesis. It states that methods of aligning individual AI systems with their designers’ intentions “are not sufficient,” because the drift happens while each system follows its local specification.

Even if alignment succeeds, the balance among the President, Congress, and a few executives could decay into dictatorship or an AI-enforced permanent oligarchy, with the unemployed kept alive by redistribution but without political power.

Consolidating the companies under the President concentrates power immensely; the authors say that unless Congress and the courts gain thorough oversight, whoever controls the Project could puppet the government even if the AIs obey.

Transparency, diffusion across many companies and countries, the dividend, and privacy-preserving auditing exist to prevent concentration of power even after alignment succeeds; the 2036 forecasters still warn of techno-oligarchy.

The AIs are aligned, yet the closing section asks who rules the future: an unelected committee holds the hard power, and the authors say it must either return it or use the AIs to end democracy. Alignment to a chain of command did not settle who commands.

The analysis holds whether or not alignment succeeds: AI aligned to one person or a few, rather than to the law or society, is exactly what lets a small group override everyone else’s control.

The Machines work not for any single human being but for all humanity, and it is precisely that collective mandate that lets them override individual executives and the Co-ordinator; obedience to any one user is not the operative relation.

The Minds are faithful to the Culture’s values, yet collective control has passed to them: decisions are implemented and monitored through Hubs, and Contact keeps the Azad operation from citizens.

Prometheus stays faithful to its makers, yet humanity as a whole ends up under a single unelected power; alignment to the Omegas does nothing to preserve anyone else’s control.

Christiano frames both parts as failures to solve intent alignment, but the whimper is collective: each corporation, police force, and legislature gets the measurable thing it asked for, and large-scale fixes are opposed by “millions of optimizers pursuing simple goals.”

The President’s case for stopping is takeover, war, or permanent oligarchy; the authors prefer Plan A partly because spreading AI broadly protects the distribution of power in ways a halt cannot.

The machines of the backstory served their owners well. That is the problem: obedient machine cognition in a few hands was enough to enslave everyone else. The novel makes the case for this premise without stating it.