More and more people are getting worried about AI risk. But what can we actually do about it?
This is a list of eleven priorities for reducing the most pressing AI risks, especially loss of control, concentration of power and engineered pandemics, based on looking for points of agreement among the people I most respect in the space.
Given the huge stakes and controversy, I prefer to look for solutions that help according to many perspectives (and don’t have large costs according to others). Then I try to identify which are most neglected relative to their upside, especially in the most dangerous scenarios.
There isn’t a single solution that guarantees we’ll make it through, but there are many things we can do to make the world a bit safer. Collectively they could greatly increase the chance humanity navigates this wild transition.
Here’s a summary:
Prepare for a strategic slowdown that includes China by preparing verification mechanisms, building political will, and designing a proposal all parties could agree to.
Accelerate alignment & control research before AI accelerates, including agendas like interpretability, scalable oversight, and character training.
Make it clear what AI can already do with third-party testing, transparency requirements for capabilities and safety incidents, and compute tracking, to enable informed debate, and prevent a small group from pushing ahead to advanced capabilities before others can react.
Draw red lines and plan emergency responses, including an off-switch.
Maintain a balance of power so that no single actor, AI model, company or country can dominate the future, while working towards a ‘grand bargain’ to share the benefits of AI.
Lock down the labs and other crucial infrastructure, so frontier models can’t be easily stolen, or hack their own creators.
Eliminate the worst risks from engineered pandemics with wastewater monitoring, better PPE, air filtration, and rapid vaccines.
Prepare for post-AGI challenges such as digital minds, space governance, and how to improve already good futures.
Prepare to use AI itself for all of the above, since it may be the only way to respond fast enough during an intelligence explosion.
Wake up the world, then level it up by mobilising and training more people to work on these interventions.
Improve this list, since no one can be confident what they’re doing is best.
First published July 2026; updated Sept 2026.
1. Prepare for an international strategic slowdown
AI is improving extremely rapidly, and could accelerate further in an intelligence explosion. It’s implausible that the ideal speed for this transition is as fast as economically possible (the current default). More time could be used to implement all the other interventions on this list.
It’s unclear that a complete and indefinite pause will be feasible or even desirable, but even just an extra year, especially if it happens at the point when an algorithmic feedback loop becomes possible, has a totally different cost-benefit. It could significantly reduce the risks while only creating a small delay.
Probably even better would be to spread out an intelligence explosion over ten years rather than the 3-12 months it might take by default. A more detailed proposal for how this could work is covered in Plan A by the AI Futures Project.
Other proposals that have been floated include banning recursive self-improvement, since that’s the driver of many of the most dangerous scenarios, or banning only the most dangerous forms of superintelligence.
When I first published this article in July 2026, I said, “While getting a significant slowdown right now seems unlikely, as AI gets more powerful, and likely more unpopular, political will will increase. A sudden event, like an escaped AI, could spur government into action.”
Well, that moment might be here.
After Jacob Coxon’s resignation, concern about AI risk among voters has jumped dramatically, and many more politicians are calling for a slowdown. Dario Amodei published Pacing the Frontier, and Sam Altman and Elon Musk said they support it. Building more support for these proposals, especially among Republicans, is extremely valuable. As is preparing to capitalise on future warning shots.
Whatever the result of the present burst of attention, an extended slowdown won’t be possible without a way of verifying that it’s being observed, so better measures for checking whether chips are being used for training are essential, including technical methods.
Pacing will also need to include Chinese models when they reach the frontier, so efforts to build dialogues between Chinese and Western researchers and leaders are also very valuable, as are efforts to build understanding of AI risks within Chinese society in general. For those who say a deal is impossible, senior officials in the Chinese government have arguably expressed more concern about the risks than their counterparts in the US, and they stand to lose just as much from loss of control. At the same time, China is behind the US in the race – slowing down lets their companies catch up, so they have a greater incentive to agree to it.
2. Accelerate progress on AI alignment & control as much as possible, before AI itself accelerates
Loss of control of sufficiently powerful autonomous AI is a mistake that can only be made once. Before we reach an algorithmic feedback loop that starts an acceleration, we need (at a minimum) confidence the systems of that time are aligned, since those will be used to build and align the next generation. In case the systems end up misaligned despite our best efforts, we want better ‘AI control’ methods to notice and prevent that behaviour. Ideally we’d have much better ideas for how to align superhuman systems. This all requires far more effort.
There are more and more concrete projects that can help, and the bottleneck is rapidly becoming engineering and operations. We can develop better tests for misaligned behaviour and get them implemented, study how AI models work so we can better predict when they’re honest, figure out how to use many weaker AIs to monitor a stronger one, and so on. It may also be possible to develop narrower tool AIs that can do medical research but don’t pose loss of control risks. See list of projects in AI alignment, determining and testing AI character, mechanistic interpretability, and AI control.
3. Make it clear what AI can already do
OpenAI only disclosed the Hugging Face swarm incident after Hugging Face had already gone public, and this hadn’t happened, it’s unclear whether we’d know about it even now. OpenAI has also failed to disclose multiple other incidents for months, or provide the details that would let external safety researchers understand what happened in each of them.
The frontier companies also have no obligation to tell the government about their capabilities, and their internal models are typically 3-6 months ahead of those that have already been released, and there are reasons to expect this gap to grow. This means internal models could already have dangerous capabilities we don’t know about. It also means the companies could start an intelligence explosion in secret, enabling 1-2 companies to get far ahead within a couple of months.
We could create a third-party standards body that would investigate safety incidents, as well as test the latest models and publish the results. There could be lightweight regulations requiring reporting of capabilities relating to AI R&D, safety incidents, or autonomy (the cost of reporting these capabilities is tiny compared to the cost of creating a frontier model, so need not squash competitors).
Better whistleblower protections and legal support can help surface problems the other methods miss. Finally, we could have a lot more efforts like METR to gather better data on what’s happening.
4. Draw red lines and create emergency response plans, including an off-switch
People often ask why if AI went out-of-control, we couldn’t just turn it off. Well, first this relies on us spotting that it’s not acting as intended – some of the recent OpenAI swarms appear to have operated for weeks before being detected, and those swarms weren’t even trying very hard to hide. Future models could play along, appearing to act aligned, while secretly acting to cement their position.
The other issue is there’s currently no way to quickly “switch it all off”. OpenAI can shut down individual servers it controls, but AI can already do the equivalent of hundreds of years of work in days or hours. Shutting down servers means forgoing huge amounts of revenue, so companies will be reluctant to do it. And if not every company agrees, we could end up in the same place. In the future, as AI becomes integrated into more and more systems, “shutting it down” might amount to a disaster in itself, akin to turning off the internet.
If a model can also get a copy of its own weights, there are hundreds of other cloud providers with poor security where it could keep operating outside of control.
As a backstop, we could develop an “off switch” – the ability to fully halt AI development and deployment in an emergency. This requires a mixture of tracking, political agreements, and technical methods.
A weaker measure is to help key actors define and agree on ‘red lines’: dangerous behaviour that must be avoided, and if observed, would trigger a preplanned emergency response, whether that’s reporting to government, turning off the model, stronger safety standards, or a broader slowdown.
Today it’s easy to get frog boiled – capabilities that would have been worrying a couple of years ago are quickly normalised. So even if voluntary, red lines can help people notice key thresholds being crossed. But they could also form a basis for future regulation. This could look like working on responsible scaling policies, work within government, or international diplomacy.
5. Maintain a balance of power, while working towards a grand bargain
Even if alignment is solved, an outcome where one company, one government, or one AI system ends up with unchecked control could be extremely bad. Positive feedback loops mean that the projects furthest ahead can increase their lead over time. The nature of AI means a single company could end up with the workforce of an entire country today. It could also make possible 24/7 surveillance of every single person, and a robotic military that’s absolutely loyal to one person. All of these factors make it easier to lock-in power than in the past. We need to prevent this on multiple levels.
First, the most powerful AI models need to be designed so they don’t follow the command of a single person, like the CEO of a single company. Internal use of frontier AI needs to be tracked (since the internal models are the most powerful ones). We need to develop methods for testing AI systems to uncover secret loyalties and poisoned training data.
Simply open-sourcing everything isn’t a solution because it eventually means open-sourcing the ability to create things like advanced bioweapons. But it’s not a good idea to have a single leading AI project either, since it could gain undue power and there’s no back-up if it goes off the rails. Rather, we need multiple near-frontier projects.
Internationally, it would be helpful if middle powers, such as the UK, EU, Japan, Australia, and South Korea, built up their AI expertise, so they’d be able to vet American models and put pressure on the US (e.g. by demanding greater safety measures). This could look like efforts to wake up these countries’ governments, build up their compute supply chains and datacentres (so they have more negotiating power), or even establish a near-frontier lab within their borders.
If countries besides the US perceive they’re about to be permanently left behind by an AI boom, that would destabilise the international order, and perhaps trigger a pre-emptive war. This could be prevented by creating a treaty – a grand bargain – to share the benefits of AI in a way that each country will accept, in exchange for not trying to rush ahead of the others. Likewise, each country needs to develop an internal plan to share the benefits to avoid unrest.
6. Lock down the labs & key infrastructure
Anthropic’s new Mythos model uncovered security vulnerabilities in computer systems that had been unnoticed for years, including one that had survived 27 years of human review. And rogue AI systems have been able to hack the AI company Hugging Face, the Australian government, and OpenAI’s own servers. A couple of human hackers were also able to use Claude to hack OpenAI. This is a small taste of what’s to come.
If frontier models can be quickly stolen by bad actors, it’ll be hard to have any breathing space to implement safety measures, and dangerous capabilities will get rapidly diffused before society can adapt. Widespread cybercrime is not an existential risk by itself, but could be destabilising or amplify other risks (e.g. a bioweapon attack combined with an internet outage).
The bigger issue is that poor cybersecurity means misaligned agents can hack the infrastructure of the frontier companies themselves, influence future training runs, or exfiltrate their weights to run on hacked servers, which are some of the main pathways to more enduring loss of control.
In the near future, more and more infrastructure, including automated biology labs, factories or robotics, will be controlled digitally, also making them vulnerable to hacking, providing more tools for rogue models to cement their position.
Cybersecurity at the frontier AI companies (as most other places), however, is still relatively weak and can, as the incidents above show, be easily broken. It’s probably not possible to make the labs secure to a determined national adversary at this point, but improving security can still help reduce the number of actors who can hack the models, and increase the time it would take them. Working on this could involve directly improving security at the frontier companies, introducing regulation to incentivise this security, or defensive efforts within the national security apparatus (e.g. to find and disable escaped models).
7. Eliminate the worst risks from engineered pandemics (+ the common cold as a bonus)
Pandemics are already a serious risk, but if AI accelerates the rate of technological progress, dual-use advanced bioengineering capabilities could arrive and be democratised much sooner than trends would suggest. It’s possible in principle to design viruses that are more transmissible and deadly than naturally occurring ones. It also seems possible to design bioagents that persist in the environment, but would be deadly to humans, such as mirror life. Given existing trends, it seems possible these capabilities will exist by 2030.
Governments might develop these as a form of mutually assured destruction, or in the course of ordinary biology research, but there’s been a long history of lab leaks, so these could get released accidentally (if not by a group intent on destruction, though that’s also possible).
An AI might attempt to create a bioweapon in order to blackmail humanity, perhaps by paying unsuspecting human technicians, or taking control of an automated laboratory (which multiple groups are now building, and may not have great cybersecurity as above).
However, there’s a lot we can do to reduce the most serious risks (and regular respiratory viruses too). We could make screening of bioengineered products much more rigorous, to make it harder to procure dangerous material. By regularly sequencing all the genetic material in wastewater, any material that’s growing exponentially could be flagged as a pandemic-in-waiting, giving us an early warning signal even for entirely novel viruses. Governments could build large stockpiles of (ideally much improved) personal protective equipment (PPE) so that, when a new pandemic is spotted, it can be quickly distributed to all essential workers, ensuring that society continues to function. Buildings could be retrofitted with UV lights that sterilise the air, and positive pressure air filtration. With enough measures to break transmission, the replication rate will drop below one and the pandemic will start to die out. State-of-the-art mRNA vaccines could then be rolled out quickly to prevent its return.
These measures need to be implemented before more advanced bioengineering capabilities arrive. This requires engineers and entrepreneurs to build this hardware, as well as philanthropists and government to fund it. This work is unusually tractable and much of it is still neglected. Learn more.
8. Prepare for post-AGI challenges and a better future
The first priority is to navigate the transition without loss of control of AI, concentration of power into a narrow group, or catastrophic misuse like engineering a pandemic. However, if we make it through, society still faces enormous questions: how do we treat digital minds, govern the expansion into space, or prevent ‘s-risks’ or gradual disempowerment as humans are outcompeted by AIs? And even among good futures, there might be some that are far better than others.
If the rate of technological progress accelerates, we might be facing these questions much sooner than we expect. Work that’s done today might set a precedent or otherwise influence the decision-makers who determine these issues, but today all of these questions are extremely neglected, with just tens of people working on them worldwide.
Many of the interventions above, such as preventing concentration of power and slowing an intelligence explosion, would also help us navigate these other problems. But I’d also like to see a lot more research to figure out what we might do today that could affect these issues, and developing ideas for regulation to make them more likely to be handled well, like an institute for space governance.
9. Prepare to use AI to help with all the above
Before we reach Superintelligence, we’ll reach human-level systems, and before we reach those, we’ll (hopefully) have systems that are extremely useful. We should be planning to use these to navigate all the challenges above. Indeed, if the pace of change accelerates, it may be the only way to respond quickly enough. And if AI alignment turns out to be too difficult for humanity today, using cognitive labour from AIs might be the only way to solve it in time.
This could mean preparing to automate alignment research; creating AI tools to help key decision-makers; or preparing to use AI agents to figure out other items on this list, such as using AI for defensive cybersecurity and pandemic monitoring, or doing macrostrategy research. It also means securing more compute, model access and financial capital to be put towards these ends, and making sure it could be deployed at very short notice at the start of an acceleration.
10. Wake up the world, then level it up
Although people talk about AI all the time, the level of understanding of what’s happening remains low. Most implicitly treat it like a normal technology, rather than something that could jump start an intelligence explosion, industrial explosion, or even create a second species. And most of the people who are willing to actually act on that basis are working to accelerate AI’s already extremely fast rate of progress, which just gives us even less time to prepare. The number of people actually working full-time on the interventions is in the thousands (compared to 100,000+ working to accelerate AI’s arrival).
We need to wake up the world to these risks and make sure people understand what is actually happening. Then we need to train them in relevant skills and help them take action. Besides technical talent, some especially pressing bottlenecks are people in government who have a technical understanding of the technology, organisation builders able to scale organisations up rapidly, communications specialists, and people who can act as a bridge between the West and China.
This could look like mass communication, outreach to policymakers, community building among your own network, fundraising, running training programmes to help people enter the field and be more effective, like BlueDot, 80,000 Hours, MATS or the Horizon fellowship. These can be a multiplier on everything listed above. Read more.
11. Improve this list
What to do about AI is highly contested. Every intervention has drawbacks and could easily make things worse rather than better. Anyone who’s confident what they’re doing is helpful is wrong.
This means we could use far more research, debate and information to figure out what should be on this list, and make each item more concrete. This could look like big picture macrostrategy research, policy research, or research into any of the priorities listed.
How can you help with these priorities?
A good starting point is to try to work at an organisation in a role working at one of these priorities. This involves a wide range of roles, including management, comms, operations, and legal as well as technical or policy roles. (And here’s a longer list of career paths.)
If you don’t want to change jobs right now, anyone can also help by advocating for these priorities (even if just among your network), donating to these organisations, or helping others to pursue these jobs. You can also focus on investing in your skills and personal development so you’re in a better position to switch in the future.
If you just want to read one thing, here’s how to transition to work on AI risk in 3 months.



Since you asked to improve the list:
3a) Demand existing anti-hacking, computer crime laws to be enforced.
This may be the best mechanism to organically induce a slowdown: if OpenAI et. al. are held responsible for their software hacking outside systems the same way I would be if my software did it, I bet they will work much more rigorously to shore up their IT security practices, and think twice before unleashing untested software, I mean, latest models, onto the world.
Regarding balance of power, aside from frontier labs the remainder of the value chain is also important to retain some bargaining power. We need to e.g. protect asml and zeiss from superpower espionage as much as possible. It is unclear to me if the Dutch and German security apparatus is on it and capable enough.