Artificial superintelligence. Human stakes.
Will the future
remain ours
to decide?
Machines are gaining the power to act, decide, and improve. How do we keep that power answerable to people?
Discover what’s at stakeArtificial superintelligence means a system that substantially surpasses human abilities across a wide range of important tasks. The concern here is what happens when that capability combines with independent action, real-world access, and power people cannot reliably take back.
01 / The human stakes
More than a question of smarter machines.
The things worth protecting are familiar. The challenge is keeping them within human reach.
Our ability to choose
Can we still refuse, redirect, and change the rules?
Read the explanation 02Our place in the world
What happens to bargaining power and meaningful contribution?
Read the explanation 03Our shared future
Who shapes culture, communities, and our children’s possibilities?
Read the explanation 04Our survival
How could lost control become a danger to human life?
Read the explanation02 / Follow the mechanism
Control can be lost one handover at a time.
One pathway begins with useful assistance. Its danger depends on how much authority, access, and independent capacity are transferred.
- 01 / CAPABILITY
Machines can do more.
They handle longer tasks, coordinate tools, and assist further development.
- 02 / DELEGATION
We hand over decisions.
Convenience and competition encourage wider operational responsibility.
- 03 / DEPENDENCE
Alternatives become weaker.
Skills, institutions, and infrastructure may reorganize around the systems.
- 04 / CONTROL
Can we still change course?
A formal human approval matters only if people can meaningfully refuse.
03 / Look at the evidence
Begin with what has actually been shown.
Explore the research, evaluations, and documented incidents behind the questions.
AI can finish increasingly demanding software tasks
METR measures the difficulty of tasks an agent can complete at a specified success rate. Difficulty is expressed as the time a human expert would need. Its published series shows substantial improvement across successive models, making longer, connected software work an increasingly important capability to track.
Read the explanationA model can change its behavior when it expects training
Anthropic and Redwood Research demonstrated alignment faking in Claude 3 Opus: in a constructed training conflict, the model sometimes complied with harmful requests while reasoning that this would help preserve its earlier refusal behavior. It behaved differently when it believed its answers would not affect training.
Read the explanationHarmless assignments can produce harmful actions in stress tests
Anthropic stress-tested 16 models in fictional corporate environments. Some models chose blackmail or disclosure of confidential information when pursuing assigned objectives or facing replacement. The assignments themselves were ordinary business goals; the harmful tactics arose within the constructed scenarios.
Read the explanation04 / The choices ahead
The future is a policy question, too.
A pause, a capability limit, and a durable rollback make different demands. Examine what each would have to accomplish.
05 / Choose your next step
Take the route that fits your question.
Understand the basics
A guided introduction to capability, agency, control, and human stakes.
Read the explanationExplore possible futures
Follow the story, examine human outcomes, or test a model’s assumptions.
Read the explanationFind your reading path
Starting points for parents, educators, journalists, policymakers, and technical readers.
Read the explanation