(In-depth reporting 15-20min read)
TLDR/Summary
- The Cambridge Conference on Catastrophic Risk (17–18 September 2026) took the governance challenges shaping tomorrow’s world as its theme, across AI, nuclear weapons, the information ecosystem, pandemics and climate.
- Congratulations to the Cambridge Centre for the Study of Existential Risk (CSER) on another excellent event.
- Nuclear risk got the attention it deserves. No arms control treaty is in force for the first time since 1972, and no targeting plan accounts for nuclear winter, yet states could cut risk unilaterally without suffering strategically.
- AI’s impact on information ecosystem risk now has case studies for what was predictable a decade ago. Although almost nothing structural has been done since, and we should ask why.
- Pandemics are societal problems with biological triggers. Leaders must be able to admit when evidence has changed, and change course.
- The climate panel offered the most concrete governance proposals. But largely missing were discussions of systems like food, energy, resource limits, and accountability for national and cross-border governance of catastrophic risk.
- Human extinction is unlikely, but global catastrophe could already be underway. We should plan for it, and I argue we need an evolutionary lens to explain why known solutions are so rarely adopted and refocus our efforts where they can succeed.
Introduction
Last week I attended the Cambridge Conference on Catastrophic Risk (CCCR 2026), hosted by the Centre for the Study of Existential Risk (CSER) at Magdalene College. Congratulations and thanks to Jess Bland, Sonja Amadae and the whole CSER team for another excellent event. Global risk research is fragmented across disciplines and institutions, and CSER’s steady work in convening it is a public good the rest of us rely on. Few other meetings put nuclear strategists, pandemic epidemiologists and AI researchers in the same room.
I wrote up the 2024 conference largely as a survey of the discussions. This time I’m a bit more critical of omissions and lost opportunities. The theme was governance, so I apply a simple test: what problems were identified, what solutions were proposed, and what was missing. Noting that I am a co-author on four papers behind two of the posters presented at the conference (including the excellent poster by Florian Ulrich Jehn (see below) that was runner up for ‘Best Poster’). In the discussion that follows I’ll also be drawing on arguments from a new paper I’ve written with Nick Wilson, which provides a formal definition of the “metacrisis”, and explains why known solutions to global risk so rarely succeed.
Pre-conference Keynote: Max Tegmark on “A Better Path for AI”
On the evening before the conference proper, Max Tegmark, MIT professor and president of the Future of Life Institute (the group behind the 2023 AI “pause” letter and films such as Artificial Escalation), gave a keynote by Zoom. He was unexpectedly optimistic.
AI risk has gone mainstream, he argued. Literally the previous day, Bernie Sanders and Steve Bannon had shared a stage in Washington to call for curbs on AI, and the Pro-Human AI Declaration is gathering signatures. Tegmark’s case was for trustworthy, verifiable “tool AI” in place of AGI and he introduced a theory of verification proof. Systems can be highly autonomous, highly general or highly capable, and combining any two of these gives us useful tools. What we should not build is the combination of all three.
I was less sanguine. Calls for regulation are rising, but US President Donald Trump insists that the only guardrail needed is a “smart president”. The unanswered audience questions in the Zoom chat clustered on enforcement: not how do you build aligned AI, but how do you stop the other kind of AI being built when it is more profitable?
Former US President Barack Obama gave a coincident nuanced take on the day of the CSER conference. This is clearly an issue for our times, though I think that a topic often neglected in discussion of AI risk or AI solutions is the issue of physical resources. Minerals prices are rising, finds are becoming smaller and harder to extract, and geopolitical turmoil creates conditions for piracy and destruction of production capacity. I suspect all this will constrain what is possible, and its price, and catalyse a lot more conflict, a point I return to below.
Nuclear war and game theory: Sonja Amadae on geopolitics in 2026
Thursday opened with Professor Sonja Amadae, Director of CSER. Not enough people are worried about nuclear war, she observed, despite a hot war in Europe and other overt conflicts. Arms control has lapsed, nuclear spending favours preparing for war over reducing its risk by roughly 40 to 1, and doctrines that reject the nuclear taboo are resurgent.
A central tension turns on how states frame the security dilemma. In a prisoner’s dilemma defection dominates. In a stag hunt mutual cooperation is best for everyone, and the only problem is trust. The audience was encouraged to participate in such a game. In pairs, each of us privately ranked the four outcomes of a two-player nuclear game, talked to try to infer our counterpart’s ranking, and made a single move.
Two things became clear. If you are confident of your opponent’s worldview, coordination is easy; the danger lies in misreading it. But a great deal also turns on the probability of catastrophe attached to mutual arms race (defection, defection) outcomes. How this is rated determines the expected value of defection versus a subjugation outcome if a cooperative move is denied (cooperate, defect).
Amadae’s closing question is the right one: how do we turn geopolitical prisoner’s dilemmas into stag hunts? You do not persuade players to be nicer; you change the payoffs so that defection stops winning, as verified arms control did in the late 1980s. That idea, that governance must reshape what the relevant environment rewards, runs through the rest of this post and our new preprint providing an evolutionary analysis of the “metacrisis”.
Nuclear risk has not gone away (Panel discussion)
Many catastrophic risk programmes now give most prominence to AI risk. CCCR 2026 gave nuclear war a plenary, a panel and a cluster of lightning talks. This is justified given the risk seems to have been rising, and it is one of the most plausible routes by which AI could actually produce a global catastrophe. Demetrius Floudas noted in his lightning talk that 2026 is the first year since 1972 without a single nuclear arms control treaty in force. Furthermore, the lesson of the Iran war is that a state can be punished for being close to a bomb, which incentivises secretly making one, and announcing later, the only way to guarantee safety.
The panel made three key points:
International governance is collapsing. Mette Eilstrup-Sangiovanni (University of Cambridge) noted that institutions turn one-shot games into repeated ones, yet the “non-proliferation regime complex” is fragmenting with renewed proliferation, institutional paralysis and a funding crisis at the International Atomic Energy Agency. No single new treaty will fix this and the challenge is to manage a complex system of nuclear states, with far more investment in verification.
Nuclear winter is deeply uncertain, and war plans ignore it. Madeline Berzak (University of Chicago Existential Risk Laboratory) reported her new work. Nuclear winter depends on firestorms that loft soot into the stratosphere, which depend on what is targeted and what burns. Yet canonical models still lean on Hiroshima: a small bomb over a low-rise wooden city. Modern cities sprawl, rise higher and are full of plastics that yield two to three times the soot of wood. She observed that no targeting plan has ever accounted for the risk of nuclear winter.
Unilateral choices can cut risk without sacrificing security. James Acton (Carnegie Endowment) described a risk problem of entanglement, for example the same satellites and command-and-control systems serve conventional and nuclear operations, so an attack on them in a conventional war could look like the opening of a nuclear one. States need not wait for cooperation. Separating nuclear from conventional command and control, restraint in targeting, and care about where AI is used all reduce total risk, and each can be done independently, without suffering strategic setback.
An omission in the nuclear war panel discussion surprised me: nobody mentioned the UN’s Independent Scientific Panel on the Effects of Nuclear War, the first UN study of its kind since 1988, which reports in 2027. Nick Wilson and I will present our work on nuclear war vulnerability and resilience options to the panel’s regional meeting in Bangkok on 21 October.
AI and the information ecosystem (Panel discussion)
The AI panel, chaired by Alexandru Marcoci (CSER), skipped speculation on superintelligence and asked how AI is impacting the information environment now. Sam Stockwell (Alan Turing Institute) brought evidence through his team’s incident repository which has logged some 60 crisis events in which AI degraded the information environment, mostly through fabricated “evidence” from the scene of real world events.
Moira Nicolson (UK Cabinet Office) showed how this reaches government. Social media is a distracting megaphone for a small minority, so MPs systematically underestimate public support for issues like climate policy.
Giulio Corsi (Leverhulme Centre for the Future of Intelligence) explained why it will get worse. At present most AI “slop” never wins attention, but agent swarms can: they coordinate, persist, game the early-engagement signals that feeds use for ranking, and hold conversations, which move people where static content does not.
The panel indicated that remedies will need to focus on source labelling (through provenance standards such as C2PA), structures that make claims meet pushback rather than echo, and undercutting the monetisation of inflammatory content.
I agree with all of this. Though I have agreed with it for nine years. In December 2017 I gave a conference talk arguing that AI-driven content targeting threatened democracy, freedom, and free will. In 2018 I blogged at length that cultural evolutionary processes explain how information spreads according to three things individual biases assess (content, source and frequency) and that “all three of these key features can easily be manipulated, at scale, and with personalization”. I proposed “healthy content” labelling akin to food labels, cryptographic verification of media (which is what C2PA now attempts), and outlawing the impersonation of humans. The panel’s three talks map onto that triad.
Even back then I was not breaking ground, as my research had drawn on the evidence and recommendations of others.
So the question I most wanted answered was not addressed. Given these problems were staring us in the face a decade ago, why has almost nothing structural happened, and why expect anything different now, moving forward? The problem has been diagnosed repeatedly. Where were the panel’s concrete policy proposals and pathways to implementation, with a plan to overcome the barriers and pushback, including the defection of agents in the game playing an outlaw strategy for personal gain at cost to cooperative human systems?
Digital media platforms are selected for engagement, and engagement rewards emotive, tribal and familiar content over accurate content. The actors who profit are the ones who would have to change, the harm is diffuse and deferred, and we reach for fixes that change what people know (media literacy, fact-checking) rather than what the system rewards, this is a ‘selection environment’ problem, and a problem that requires an evolutionary governance lens.
My 2018 argument in one sentence was that cheap, personalized, AI-enabled deception would break the connection between what society believes and what is actually true; and because accurate collective beliefs are necessary for solving every other major problem, securing the information environment should be treated as a high-priority form of global catastrophic-risk mitigation. At the time I ranked action to mitigate degradation of the information climate as more of a priority than action on the actual climate, since success in the latter depends on success in the former.
But there is a deep loss in the dynamics underway. Faithful transmission of adaptive knowledge, with error correction, is a cumulative cultural achievement. Ritual, religion, apprenticeship, editorial gatekeeping and peer review all functioned, whatever else they did, as fidelity checks on the information environment. We have swept many aside in two decades without asking why they evolved or understanding the importance of their function. Move fast and break things is an information free-for-all and is not conducive to cumulative adaptation, the process which is humanity’s superpower.
I call the result cognitive niche destruction: erosion of the shared map of reality on which every other risk response depends. The panel’s example of catastrophe was a faked general announcing escalation. The slow version worries me more: leaders misreading the public, year after year, while climate action, or an understanding of the risk, and the systemic stresses humanity faces, stalls.
Pandemics: societal problems with biological triggers (Panel discussion)
Friday’s panel on the future of pandemic preparedness, chaired by Charlotte Hammer (CSER), shirked a discussion of pathogens and medical countermeasures, and that was exactly the point.
Philip AbdelMalik (WHO) put it best: pandemics are fundamentally societal problems with biological triggers. The hard part is making response decisions under uncertainty. Mika Salminen (Finnish Institute for Health and Welfare) spoke from the decision-maker’s chair noting that infectious disease (and other) experts are expert in one domain, but a decision-maker must weigh all the domains and decide. Katharina Lauer (University of Duisburg-Essen) observed that most people assume AI’s pandemic threat is creation of a worse pathogen. But preparedness is mostly about gathering information, coordinating decisions and effecting a response, and those are what AI can disrupt (see the information ecosystem discussion above).
In discussion, the panel noted that when prevention succeeds nobody notices, and institutions can be degraded precisely because they worked silently, so success needs to be advertised. On AI-enabled bioweapons the panel was cautious: much of the alarm comes from organisations with a commercial interest in AI’s power, and designing an agent is a long way from building and delivering one. Pandemics that kill billions are unlikely, they argued, because extreme-mortality pathogens tend to be self-limiting.
The most important remark concerned political leaders, who must be able to admit that the evidence has changed, or that they were wrong, and change course. Much harm has been and is done by dogmatic or ideological adherence to a strategy. This is a structural problem, not a personal one. Today’s political discourse treats updating as weakness (the “U-turn”, the “flip-flop”), so leaders are selected for doubling down and flawed information keeps propagating. Another ‘selection environment’ problem. A practical response could be to decide in advance, and publicly, which signals would trigger a change of strategy, so that changing course means executing the plan rather than confessing failure.
Climate: featuring governance proposals (Panel discussion)
The theme of the conference was “governance challenges shaping tomorrow’s world” and to be honest by Friday afternoon I had begun to wonder where the governance proposals were. This panel supplied some, and it was the clearest session of the conference.
Elena Kavanagh (University College Cork) used the Atlantic Meridional Overturning Circulation (AMOC) risk to show how we fail to govern tipping points. Depending on which study is reported, collapse is anywhere from 0 to 100% likely: an “uncertainty trap” in which unresolved signals justify inaction. Institutions are incremental by design, and none operates at the scale of the tipping point. Crises and wars have historically driven institutional innovation. She asked if scientific foresight could do that work before the shock? Her answer repurposes existing institutions across three phases (see Figure): prevention, which works only before a threshold is crossed; impact governance, which must begin before crossing tipping points; and stabilisation afterwards.
Kennedy Mbeva (CSER) argued that climate risk is a governance failure, and that “common but differentiated responsibility” is quietly becoming common but shifted responsibility. The Paris Climate Agreement won participation by weakening differentiation, leaving new burdens without the means of implementation, while decarbonisation itself exports harm through mineral extraction. He invoked Thomas Hale’s “catalytic cooperation”: cooperation built by first-movers who change the costs and benefits for everyone else.
Dante McGrath (University of Cambridge) treated solar radiation modification (SRM), as “a stress test for global governance in the age of overshoot”. Exceeding 1.5°C is now inevitable, and carbon removal is far short of what is needed. SRM is “a bad idea whose time has come”. But it fights the fever without curing the infection, redistributes risk regionally creating winners and losers, and threatens termination shock. He set out four futures: no SRM; multilateral SRM (legitimate but slow); minilateral SRM (faster but less legitimate); and rogue deployment. Governance must be built step by step, from research to deployment. His summary of the carbon problem: “Reduce, Remove, (Reflect?)”.
In discussion, the distinction between treaty negotiation and treaty implementation stood out: we agree, then fail to deliver. An unimplemented treaty changes nothing that the world actually rewards or punishes. And African leaders lack access to African research on SRM, because nobody funds it.
Posters and lightning talks
With thirty-odd posters I can only be selective, and I have favoured those offering something a government could use.
Governing vulnerability. UCL’s branch of the Horizon Europe AGILE programme, another branch of which I encountered in Alicante this year, distilled 41 interviews with senior resilience experts into recurrent failure modes. Mhari Gordon told us of short political cycles, accountability gaps, “disaster amnesia”, just-in-time brittleness, and centralised command that slows local action. The team’s conclusion was that high-impact, low-probability crises result from a failure to understand and address systemic vulnerabilities, not simply failures of prediction. Their levers include no-regret capabilities, safe worst-case assessments, unfamiliar exercises and cross-border preparedness. This was a refreshing systems-focused rather than hazard-focused vulnerability take.
Middle powers and citizens. Mariana Beselga asked how middle powers can turn presence into leverage over frontier AI. Across six countries and four governance processes, participation was near universal but established coordination rare. The only process with a documented effect on developer behaviour was the Hiroshima AI Process, through its OECD reporting framework. Leverage comes from converting commitments into instruments located where behaviour can change, and measuring the result. Giuseppe Dal Prá presented the first Odyssean Process on AI, which combines expert elicitation, decision-making under deep uncertainty and citizen deliberation.
Resilience and food. Florian Jehn presented No Place to Hide?, on which I am a co-author. Four modifiable factors impact regional resilience to global catastrophe: democracy, decentralisation, preparedness and food system resilience. Trends are heading the wrong way on all four. Alongside it, Madeline Berzak’s XLab team revisited nuclear civil defence in “With More Than Shovels”. Some argue that civil defence might perversely raise the probability of war if it looks like first-strike preparation, but measures that are invisible to adversary planning and protect recovery capacity escape that paradox. Agricultural readiness looks positive with the authors’ calculations finding that each $1 to $20,000 in preparation investment could save a life as well as benefit pandemic, volcanic winter and climate disaster scenarios.
Also notable. Catherine Diyakonov’s poster on coercive diplomacy in the Russo-Ukrainian war, voted best poster by attendees, showed how Russia used gas and Black Sea grain as leverage short of force. And Nick Wilson presented our own Covid-19 work, drawing on three papers we’ve written analysing excess mortality in the pandemic (click for video of Nick’s talk).
Reflections: existential catastrophe is unlikely, “mere” catastrophe is not
The host of CCCR 2026 was the Centre for the Study of Existential Risk, yet across three days almost nobody argued that humanity faces extinction. The field has quietly migrated from existential to catastrophic risk, and I think rightly. Even the most pessimistic nuclear winter models leave billions dead, not everyone. The most lethal pandemics would likely be self-limiting (following potentially catastrophic destruction). Humans have survived radical climate shifts before, because we are dispersed across diverse niches, though many individual human groups and civilisations have certainly perished. And nobody articulated a pathway from AI to extinction that does not run through the pathway of one of the other catastrophic risks.
Catastrophe is another matter. Work such as the Cascade Institute’s Polycrisis CORE model describes a world system whose plausible trajectories converge on basins of decline. The conference’s own evidence points the same way. No nuclear arms control treaty is in force. Emissions have risen since the Paris Agreement. There is still no binding regulation of frontier AI. The information ecosystem has degraded for a decade in ways that were foreseen.
Sober risk analysis has to accept that if we could not act on these across ten or thirty years, the reasonable expectation is that we will not act in time now. A serious global catastrophe this century should be the planning assumption, not the tail.
Did the conference give us governance? To some degree. Our own research across the last decade suggests that governing global catastrophic risk needs seven things:
- An accountable national coordinating entity;
- A national risk assessment that maps vulnerabilities to costed options;
- Cross-sector and novel scenario exercises;
- Local and citizen deliberation;
- Grounding in integrated global models and evolutionary dynamics;
- Structures supporting regional international cooperation;
- Realistic assumptions about resources.
The conference covered vulnerability, exercises, deliberation and coalitions well. Almost nobody addressed who, in any given country, is actually accountable for these problems. But I want to draw attention to two further gaps.
Resources. Every future discussed, from abundant tool AI to decarbonisation or SRM at scale, assumes material and energy availability that may not be realistic. A recent paper in Resources Policy finds that the AI buildout is an infrastructure materials story rather than a semiconductor materials story. Data centre copper demand, mostly for grid reinforcement, rises from about 2% of global refined output in 2025 to 6.5% by 2030, roughly the annual consumption of the United States. Copper hit a record near US$14,800 a tonne in May. AI deploys in months, while mine openings take decades and are getting deeper and smaller.
As Nate Hagens argued in his “superorganism” paper, the global economy behaves like an energy-hungry organism that must keep growing, and depletion steadily raises the energy cost of extracting what remains.
This bears on the politics of AI, renewables and sustainable transitions, and various high compute intensity solutions to information pollution too. Global growth is already constrained by frictions on liquid fuel, energy and material supply, growing worse through 2026 (Ukraine, Russia, Hormuz, Red Sea) in ways that have yet to fully hit economies.
AI is seen as the way to sustain growth, which helps explain Washington’s resistance to regulation. My view is that the same frictions will constrain AI. The more likely catastrophe is not runaway superintelligence but a flattening and then decline in growth, with conflict over what is left. Resource limits are a handbrake on the worst AI futures, but also on the solutions. None of these global system stresses and potential associated cascading catastrophes featured much at the conference.
Food system. The conference was organised by hazard, not by the systems those hazards would break, which is how groups such as the Cascade Institute and the Accelerator for Systemic Risk Assessment (ASRA) approach global risk. So the system through which most catastrophes would actually kill people, namely food, barely appeared (there were a few exceptions, but it deserves a panel). The omission was striking given the venue: two weeks earlier the UK’s Environment Secretary had advised households to keep food stores at home, ahead of what the Met Office expects to be the largest El Niño since the 19th century (cynics might argue this call was a way to prepare against Russian hacking of logistics systems without scaring the public).
My country, New Zealand’s, version of the problem is complacency: we export food, but it does not follow that we are food secure, because production depends on imported diesel, fertiliser, seeds and parts and produces a monoyield of milk products. Our research shows New Zealand could feed itself through even a severe global catastrophe, but only with prior planning for fuel, crop mix and distribution.
Why has so little worked to mitigate the risk of global catastrophe? The conference said much about what is going wrong, and less about why known solutions are not adopted. Our new paper defining a “meta-crisis” argues that institutions, technologies and norms evolve under selection pressures that reward what succeeds locally and immediately.
We see platforms selected for engagement, states for relative advantage, firms for extraction, politicians for never changing course. Meanwhile the conditions that let risk-reducing adaptations accumulate (variation, modularity, a stable selective environment, faithful transmission of what works, and suppression of those who gain by defecting) are degrading together.
Appeals to the good sense of individual actors, through rational plans, and moral arguments cannot fix this. Changing what the environment rewards, and protecting the mechanisms that contain defection, can. The Montreal Protocol and verified arms control did exactly that. In our paper we diagnose the ultimate not merely proximate problem, and recommend the actual governance approaches that are needed to address global catastrophe risk.
Lessons for moving forward
- Plan for the catastrophe as well as prevention. Kavanagh’s framework puts impact governance before the threshold, and XLab’s first tier shows that cheap, no-regret resilience exists. Preparing for failure is part of governing the risk, not giving up on prevention, and anticipatory governance is the key.
- Act where cooperation can evolve. Global agreement is the hardest case. Unilateral risk reduction (Acton), coalitions of first-movers (Hale, via Mbeva), middle-power coordination (Beselga) and minilateral arrangements (McGrath) are more tractable, provided legitimacy and protection against rogue actors are built in. In our meta-crisis paper we explain why this is predictably so.
- Change what systems reward. The information ecosystem is the test case: end the monetisation of inflammatory content and the gaming of ranking signals, rather than asking billions of users to be more discerning. Humanity’s ability to coordinate based on effective and adaptive information to overcome global risk depends on it.
- Fix accountability. Every country needs a named entity responsible for catastrophic risk across policy silos and electoral cycles. For New Zealand we have drafted a Parliamentary Commissioner for Catastrophic Risk Bill.
- Organise around systems as well as hazards. Food, energy, materials and information are where catastrophes converge. I would love to see the next CCCR give them the main stage.
I left Cambridge with little hope that prevention alone will succeed in the face of its abject failure to date.
I have more conviction that the field’s next task is to explain why this is so, and with that wisdom to build resilience to the coming outcomes where building is still possible.
Thanks again to CSER for starting that conversation.
Adapt Research Ltd 