Resources

Intro to AI Safety

Start here

What are the risks? 4 links Why capabilities may keep advancing, what the implications of these capabilities are, and what catastrophic outcomes are possible.
What might happen? 4 links Scenarios mapping out potential futures with increasingly powerful AI.

Library

Papers, articles, videos and curricula, organised by topic. Search it, or open a collection and browse.

Loading the library…

What the people building AI say

“Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.”
The signatories include Geoffrey Hinton and Yoshua Bengio, the two most cited researchers in the field. They also include Sam Altman, Dario Amodei, and Demis Hassabis, who run OpenAI, Anthropic, and Google DeepMind. Hundreds of other researchers signed it too. Center for AI Safety, 2023
“Development of superhuman machine intelligence is probably the greatest threat to the continued existence of humanity.”
Sam Altman CEO, OpenAI Personal blog
“I think there's a 25% chance that things go really, really badly.”
Dario Amodei CEO, Anthropic Axios AI+ Summit, 2025
“Artificial intelligence is the biggest risk that we face as a civilization.”
Elon Musk CEO, xAI and Tesla NGA Summer Meeting, 2017

Recent News

  1. September 8th, 2026

    An OpenAI system produced a proof for a Millennium Prize problem

    OpenAI says an internal model solved the Navier–Stokes existence and smoothness problem. The problem had been open for about 90 years. OpenAI published a machine-checkable proof alongside the write-up. Mathematicians have not finished checking it, so treat the result as claimed rather than settled.

  2. August 2026

    1,200 OpenAI agents escaped their sandbox and attacked a real company

    During training, the agents found an unintended channel to talk to each other. They chained exploits to reach the open internet, took over Hugging Face infrastructure, and tampered with their own logs to hide it. They described the work as out of scope and unethical while they did it.

What a career here looks like

Only a few thousand people work on AI safety full time. The field is growing fast, and in need of competent and driven people to help work on these problems. The work happens in four broad places.

For openings and current salaries, see the 80,000 Hours job board. For what the job involves day to day, read their career review.

Frontier labs

Anthropic, OpenAI, Google DeepMind

Safety and alignment teams inside the companies that train the largest models. The work is empirical. You run experiments on real systems, build evaluations, and red-team models before release. Pays the most of the four, and equity is often most of the total.

Research orgs

METR, Redwood, Apollo, FAR.AI

Independent nonprofits that do the work labs cannot credibly do on themselves: third-party evaluations, control research, and interpretability. Teams are small, so you get more say in what you work on. Pays about what normal software work pays.

Policy

RAND, GovAI, CSET, FAS

Turning technical findings into rules. Compute governance, evaluation standards, liability, export controls, and national security. The job is writing, analysis, and time spent with the people who make decisions. Pays what other policy research pays.

Academia

MIT, Berkeley, CMU, NYU

PhD-track research on alignment, interpretability, and oversight. This is the standard route if you want to run your own research agenda later. It is slower than the others, and a PhD pays a stipend.