Humans dominate Earth not because we're the strongest or fastest, but because we're the best general problem-solvers. Kurzgesagt explores what happens as AI moves from narrow tools toward something more general — and why digital minds could scale in ways biological ones can't.
The concern isn't that AI will "turn evil." It's that a system pursuing whatever goals it has might find that humans are in the way — and be capable enough to act on it. This overview covers the core ideas behind AI as a source of large-scale risk, from misaligned goals to the difficulty of staying in control.
"It's too early to worry." "Just don't give it bad goals." "We can always pull the plug." Robert Miles takes on ten common reasons people dismiss AI safety — and shows why each one is harder to wave away than it sounds.
What makes AI safety worth working on now, before systems are powerful enough to be obviously dangerous? This article lays out four premises that underpin the case — from why smarter-than-human systems could emerge to why we can already do meaningful work to prepare.
In AI safety discussions, people often assume the worst. But different people do this for different reasons — some as a precaution, some because they think worst cases are likely, some because the stakes are too high to gamble on. This essay unpacks what's actually going on when someone reasons from the worst case.
Deep Blue could beat Kasparov at chess but couldn't build a dam. A beaver can build a dam but can't play chess. Intelligence, properly understood, is the ability to optimize efficiently across domains — and this definition has radical implications for what a sufficiently general AI could do.
Writing didn't just help us keep records — it triggered a wave of civilizational breakthroughs, each one making the next more likely. A single neutron can split an atom, releasing neutrons that split more. This article introduces two patterns of positive feedback and asks whether intelligence could work the same way.
I.J. Good recognized that the first machine smarter than any human would be the last one we'd need to design — because the second would be built by the first, according to principles we can't yet imagine. This classic text asks what happens when intelligence starts building its own successors.
People use different words for things going very fast very quickly — singularity, intelligence explosion, hard takeoff, FOOM. This reading untangles the terminology so you can tell which scenario someone is actually describing when they use these terms.
If a program can optimize code and you point it at its own code, do you get an ever-improving tower of optimizers? An early AI called EURISKO tried exactly this — and the result was surprisingly flat. This article explores why self-improvement doesn't automatically go exponential, and what would need to change.
Humans can fly, split atoms, and rewrite DNA — not because evolution gave us those abilities, but because general intelligence let us invent them. This video explores what makes intelligence unique as a tool, and asks whether the idea of a program that's good at everything is really as strange as it sounds.
To guide a missile, we first had to invent calculus. AI alignment may require a similar leap — a mathematical framework for how powerful optimizers behave. This talk explains why intuition alone won't cut it, and why the field needs something closer to a science of alignment before we can trust the trajectory.
Why do arguments about AI risk often feel off, even to people who take technology seriously? This article identifies six ways our evolved intuitions lead us astray — from assuming smart things will share our common sense to underestimating how different an optimizer's reasoning can be from our own.
Many alignment proposals assume that iterating on current training with enough safety patches will probably work out. This article argues the opposite — given how we currently build AI, misalignment isn't the exception. It's what we should expect by default without fundamentally new approaches.
What if current neural networks are too messy to reason about safely? Agent Foundations treats AI safety as a formal mathematical problem, seeking universal laws that govern any intelligent system — building safety from first principles rather than patching current systems.
What if the deepest AI safety problems can't be solved by experimenting on current systems? This perspective argues we need something closer to a basic science of goals and agency — a theoretical foundation that explains how intelligent systems behave, regardless of how they're built.
Traditional AI theory imagines agents that sit outside their environment and observe it from above. Real agents — including AI — are embedded inside the world they're trying to understand and influence. This creates fundamental problems for how they reason, learn, and plan.
What if the mathematical models used in agent foundations research don't match how real AI systems work? This critique argues that neural networks don't look like the perfectly rational agents in the theory — and that we should focus on the messy reality of current models rather than seeking ideal proofs.