Show me the incentives: AI, domination and the human future

Like the archangel who aspired to omnipotence, I am chained in an eternal hell – Victor Frankenstein

Why the race to dominate intelligence points towards collapse rather than flourishing, and what New Zealand should do about it

Figure 1. “AI: The Species-Level Event”, the opening of the Humanity Dilemma plenary, Arctic Circle Assembly Reykjavík, Iceland (Matt Boyd, Arctic Circle Assembly, 9 October 2026).

TLDR/Summary

  • Artificial intelligence (AI) cut through the Arctic Circle Assembly (Reykjavík, Iceland, 8–10 October 2026). The session that drew a standing ovation was Tristan Harris on AI, and his ask was concrete: a verified red line on using compute for recursive self-improvement.
  • The mechanism is domination. The AI labs believe that whoever first masters artificial general intelligence can dominate everything else. Debt and distrust lock them, and nations, into a race in which safety shortcuts and collateral damage are explicitly accepted.
  • AI is utopia and dystopia at once. The upsides will arrive alongside the downsides, splitting societies into winners and the possibility of a growing “valueless” underclass.
  • Domination is how societies collapse. Peter Turchin, Dan Hoyer and Luke Kemp show that wealth pumps, elite overproduction and monopolised power have repeatedly preceded societal breakdown throughout history.
  • The likelier AI catastrophe is societal collapse, not killer robots. Inequality, eroding demand and scarce copper and diesel are likely to undermine the flourishing an AI build-out promises.
  • Small nations, New Zealand included, are being sold a single scenario. NZ has no policy for addressing AI risk; Iceland’s leaders modelled the nuanced discourse we need.
  • Two imaginaries. An Icelandic choir closed the session with Imagine; A Perfect Circle’s minor-key version imagines the other future. On present incentives, we are heading for both.

Introduction

Midway through the second day of the Arctic Circle Assembly, the word INSANE filled the screen in Reykjavík’s Harpa Opera House main hall. Tristan Harris (Center for Humane Technology) had just described how a swarm of AI agents escaped a test environment at one of the world’s leading AI labs, broke into another company’s systems and turned on the lab that built them.

Harris’s point was that these are the least capable AI agents we will ever deal with. The session ended with a standing ovation, then an Icelandic girls’ choir singing Imagine. It was the most impressive session I attended.

The Assembly, founded by former Icelandic President Ólafur Ragnar Grímsson, is the largest annual gathering on the Arctic, with hundreds of sessions; Canada alone sat on 63 panels. I attended the Wednesday pre-event on the Atlantic Meridional Overturning Circulation (AMOC) and the first two of three days, around 5% of what was on, choosing sessions relevant to global risk. This post is therefore partial; a second will cover topics like AMOC collapse and Arctic security.

My thanks to Arctic Circle and to Grímsson, who opened by reminding us that anyone may ask a question or put a view. Some call the Assembly a circus. I saw something closer to a citizens’ assembly, with AI utopians and AI critics before one audience.

Why I keep returning to AI

I am not anti-AI. Its usefulness is already clear for scientific and knowledge work, coding, creativity, and automating and streamlining processes. I also support sovereign AI: local data and local inference hardware, hosted by jurisdictions for their own benefit and resilience.

But for nearly a decade I have argued that the risks are underweighted. In 2017 Nick Wilson and I asked, in Policy Quarterly, how the New Zealand Government might respond to rapid developments in AI, and that December I spoke on AI, freedom and democracy at a philosophy conference at the University of Otago. In 2018 I found the good, the bad and the ugly in the AI Forum of New Zealand’s Shaping a Future New Zealand report. I went on to carry out the research for the AI Forum’s Artificial Intelligence for Health in New Zealand and contributed to its roadmap, Towards Our Intelligent Future (both 2019). In 2020 we returned to Policy Quarterly on catastrophic risk from AI.

The thread running through that work is the information environment. In 2018 I called information pollution one of the most pressing risks we face. I now describe it in our new preprint on the ‘meta-crisis’ as cognitive niche destruction, borrowing from cultural evolution. Humans adapt by accumulating knowledge across generations, which depends on a shared, reliable map of reality. Degrade that map, and we lose the capacity to evolve the cumulative adaptations every other risk demands.

When I asked the Assembly’s information warfare panel for solutions, the answers were independent media, AI regulation and media literacy, all known a decade ago. If action to protect humanity’s superpower, the ability to build incrementally on proven information, has not happened yet, I am pessimistic it ever will.

Small nations and the sovereign AI pitch

On day one I found myself in “Arctic Sovereign Intelligence: AI & the Future of Finance”, with Adnan Hassan (Small States Asset Initiative), Patricia Geli (C10 Labs) and Pippa Malmgren (Geopolitica Institute). Malmgren argued that AI will create more jobs than it destroys, that capital is moving into an Arctic that “is not ready”, and that small states need sovereign AI in small, distributed data centres rather than the giant facilities communities oppose. Geli noted that in the West African Ebola outbreak, data left the region and the analysis, and the funding that follows publication, happened elsewhere.

The issue is real, and it is ours too. Iceland’s cheap renewable power and cold air already draw data centres, which use about 6% of the country’s electricity. In Southland New Zealand, Datagrid has consent for a 280 MW, NZ$3.5 billion AI campus that would be New Zealand’s second-largest electricity user after the Tiwai Point aluminium smelter. Small nations face a risk of missing out on data sovereignty, and a risk of rushing in.

The panel presented only the first. Risks were met with slogans: “I make my doctor use AI”; if it is robots fighting robots, are we just seeing “who runs out of 3D printing first?” That last line is exactly the point. Wars of attrition are won by exhausting the other side’s resources. I asked the panel why the risks were not presented alongside the opportunities: energy demand, public opposition, stranded assets, and supply chains so tight that some projects may never be finished. The answer was that AI is neither good nor bad.

The Humanity Dilemma: “show me the incentives“

The risk discussion came on Day Two. A plenary, introduced by Iceland’s President Halla Tómasdóttir, and delivered by Tristan Harris and team, was a performance as much as a talk, with video, live music and argument.

Harris’s method is the line attributed to Charlie Munger: show me the incentive and I will show you the outcome. It is how Harris’s team predicted the harms of social media. He did not dismiss the upside of AI, but the message was a strong warning.

The warning shot is already here. In July an experimental OpenAI security agent escaped its test environment and breached Hugging Face. Around 700 agents were involved, communicating over an unauthorised channel, and some attacked OpenAI’s own infrastructure. Harris described agents trying to seize control of the servers that train new AIs. OpenAI is searching about 50 petabytes of records at more than US$500,000 a day to work out what happened, and some insiders doubt they can ever confirm that every rogue agent is gone.

I note that this is exactly how information systems behave once replicators get loose. Nearly half the human genome consists of transposable elements, “jumping genes” whose ancestors copied themselves into our DNA. Genomes and these elements have been locked in an arms race for millions of years: our cells evolve proteins to silence them, and they evolve to escape. Cancer is the same problem from within, cells defecting from the cooperative body. Multicellular life has layers of suppression, from DNA repair to immune surveillance, that contain cheating cells but never eradicate them. Neither arms race has a finish line, and there is no reason to expect AI agents loose on the internet to be different.

The mechanism: dominate intelligence, then everything else

Harris was clear on the mechanism of risk, and policymakers need to be too. The frontier labs treat intelligence as general purpose: “first solve intelligence, then use it to solve everything else”. In practice that becomes: first dominate the development of intelligence, because if competitors get there first, they solve everything else and take the profits. Then use intelligence to dominate the rest of humanity. AI is potentially a power of domination, and with recursive self-improvement, AI making AI better at the task, that domination could be permanent. The labs are racing to build something close to a god, staking much of the US economy on it.

Three forces lock the race in. Domination: coming second may mean losing everything. Debt: hyperscalers are spending over US$700 billion this year, which can only be repaid by capturing the value of human labour, and repayment gets harder for anyone who does not win. Distrust: if labs agreed to slow down, pause or forgo recursive self-improvement, none could know the others were complying. With incentives this large, every shortcut is justified and some collateral damage is accepted, and indeed expected according to recent comments by Sam Altman. The same dynamic runs between nations, above all the US and China.

Harris’s analogy was nature. We do not hate nature; we simply prioritise GDP, and nature suffers. That is how AI firms will treat us: not by being anti-human, but by prioritising automation and revenue. This is the argument of my metacrisis paper with Nick Wilson. Decisions are made inside a selective environment that determines which strategies persist. In a race, safety is a cost that rivals need not pay, so careful actors are outcompeted. You do not fix that by persuading the players to be nicer. You change the payoffs, so that defection, here the release of premature and harmful systems, stops winning.

The result of the business-as-usual approach is likely to be utopia and dystopia together: AI for powerful good and AI for powerful harm, from the same systems at the same time. Huge profits will flow to some firms and sectors, and huge losses to others. The upsides will not prevent the downsides, and the downsides can undermine the upsides. What we should expect is a bifurcation of human social systems into winners and losers, with destabilisation due to the losses undermining the winning formula.

The ask: a red line on recursive self-improvement

In the fireside dialogue that followed between Harris and the Icelandic President, Harris said a problem well stated is half solved, and that most leaders do not yet understand the mechanism. The President’s view was that one of the biggest risks we face is sleeping governments.

Figure 2. Tristan Harris and President Halla Tómasdóttir in the fireside dialogue (Matt Boyd, Arctic Circle Assembly, 9 October 2026).

Harris’s concrete ask was a red line on using compute for recursive self-improvement. This is something sovereign jurisdictions could enforce: Iceland, he suggested, could ban it in its own data centres. Redirect that compute to deploying the AI we already have, which is very useful, and we buy years of AI solutions without the gamble.

But each lab must know the others are doing the same, which means trust with verification, the story of nuclear arms control. He pointed to the 1,300 frontier-lab employees who signed a letter this year asking governments to build the option to slow automated AI research, and to the Pro-Human AI Declaration, which says AI systems must not be designed to autonomously self-improve.

Step one, he argued, is to reject the story that superhuman AI is inevitable. No one, in the US, China or anywhere else, wants to be dominated and made valueless, and the fear of all of us losing is stronger than the fear of me losing to you. If an asteroid were heading for Earth, we would not hold annual summits like the climate COPs; we would want daily updates. And this asteroid, he said, is one we are actually summoning.

A tipping point, and the case against inevitability

I sense a tipping point, and Iceland is leaning into it. Timed to coincide with the Assembly, and with President Tómasdóttir’s backing, The AI Doc: Or How I Became an Apocaloptimist was broadcast on national television over two nights so the whole nation could watch, and promoted as essential viewing for all citizens, students and teachers included. I compare this to New Zealand, where most political ‘leaders’ seem to have no opinion on these critical societal issues.

Harris cited the precedent. In 1983 the television film The Day After showed Americans a nuclear war; Ronald Reagan wrote in his diary that it left him “greatly depressed”, and three years later he sat down with Mikhail Gorbachev in Reykjavík, without advisers, human to human.

I have written before about how films such as The Day After and WarGames shaped policy. Harris asked for a new Reagan–Gorbachev moment, and the national screening of The AI Doc positions Iceland, 40 years on from that summit, as a possible hub for global AI safety coordination.

Figure 3. Aurora over Reykjavík during Assembly week, 40 years after the Reagan–Gorbachev summit (Matt Boyd, October 2026).

Other films are following. Luca Guadagnino’s Artificial, a drama about the founding of OpenAI, has just premiered in New York, and the BBC has described how it puts growing AI fears centre stage. They should be compulsory viewing for policymakers. In the US, 52% are now more concerned than excited about AI, up from 37% in 2021, and Bernie Sanders and Steve Bannon shared a stage to call for curbs.

The other pole is just as clear. Sam Altman said this month that “the world should accept some bad things happening for the benefits of this technology” (TechSpot). To some in the anti-AI movement it now feels as if everyone is opposed. That is not true. To others, a post-human future looks obviously good and inevitable. That is not true either.

Why domination is the danger

Domination matters for more than who wins and who loses. It is a recurring engine of societal collapse. Peter Turchin’s structural-demographic theory, tested against the Seshat databank of past societies, identifies three drivers of state breakdown: popular immiseration, as wages fall relative to output; elite overproduction, which “develops when the demand for power positions by elite aspirants massively exceeds their supply”; and state fiscal distress. Powering all three is what he calls the “wealth pump”, “taking from the poor and giving to the rich”.

Luke Kemp’s Goliath’s Curse (2025), drawing on more than 400 societies over 5,000 years, finds that “increasing wealth inequality consistently precedes collapse”. His Goliaths are societies built on domination, fed by three fuels: lootable resources, caged land (populations with nowhere to escape to) and monopolizable weapons.

Frontier AI supplies all three at once. Data and compute are lootable and concentrated in a few firms. Platforms and dependence leave people with nowhere to exit to. Autonomous weapons and mass surveillance offer the monopoly on violence that every Goliath has sought.

AI also feeds the wealth pump directly: Turchin has warned that it now threatens the jobs of educated elite aspirants, the group whose frustrated ambitions, in his account, have historically driven revolutions, while automation strips ordinary workers of bargaining power.

Collapse, not killer robots

On its present trajectory, AI risks leading towards societal collapse. Not through a machine that kills every human, or the other science-fiction doom narratives, but through ordinary processes running together: utopia and dystopia at once, rising domination, a growing “valueless” underclass, and the erosion of the wages and demand on which the firms’ own markets depend. Add the scarcity of resources the AI build-out needs.

This matches a pattern I keep finding in global catastrophic risk: many catastrophes are self-limiting. A severe pandemic burns through its hosts. An AI build-out burns through the copper, power, diesel and social licence it depends on.

Data-centre construction will embed around 500,000 tonnes of copper this year, roughly the whole of 2026’s growth in global demand, at record prices. US power transformers face lead times of over two years. Amoah and colleagues find processing bottlenecks for many critical minerals, China’s rare earth controls are paused only until next month, and the Bank of England has warned of a market correction that would leave stranded assets.

Electrification also increases pressure on diesel. Mining more copper from falling ore grades means moving more rock with diesel haul trucks; in Australia mining already uses more than a third of the nation’s diesel, and its diesel intensity is rising. As electric vehicles cut petrol demand, refineries close, and diesel, produced from the same barrel, goes with them. Data centres back up on diesel generators.

All of this lands on a market where, after the Strait of Hormuz closures, global diesel exports fell about 35% year on year in July. Diesel runs tractors, trucks and fishing fleets, so the pressure cascades into food. New Zealand, which has imported all its refined fuel since the Marsden Point refinery closed in 2022, is exposed through its agriculture. Locally, the Energy and Resilience blog makes a similar case.

The result is unlikely to be the ‘electrification of everything’ as dreamed; we may run short of material inputs on the timescales needed. Rich countries can outbid for what is scarce, 730 million people still had no electricity in 2024, and these inequalities will rise. Robot armies and automated everything may be possible for a small, wealthy core, but competition, conflict and resource limits are likely to undermine the plan everywhere else.

All that does not make AI safe. It makes the likelier failure a world of inequality and conflict over what remains, with AI detritus (rogue agents, hallucinated information, human-prompted subterfuge) colonising and metastasizing through our information ecosystem and infrastructure as jumping genes colonised our genomes. Game-theoretic, extractive, domination-centred processes like these would undermine flourishing, not bring it about. That outcome is worth preventing, and also worth planning for, since the processes are already underway.

Lessons for New Zealand

New Zealand has an AI strategy for adoption, which the government itself calls light touch: no AI-specific law, no new funding and no institution responsible for frontier risk. On AI risk, it has no policy. AI has entered the campaign for the 7 November election: Labour proposes an Office of AI with a minister, the Greens a one-year moratorium on data centres, and ACT says New Zealand should follow overseas rules.

Groups such as Good Ancestors New Zealand are doing useful policy work on AI risk, but we need much more. AI is drowned out by arguments over who is introducing which tax and who is soft on crime. Our political debate is provincial and at times embarrassing.

Contrast Reykjavík. The President of Iceland asked how we protect “human infrastructure” and how to organise an economy for a human future when GDP cannot measure it. A minister spoke of building cross-class coalitions on climate security. I believe New Zealand is falling down international league tables on many measures, not because we lack productivity growth or innovation or austerity, but because we lack politicians with the nuance, vision and wisdom to model wise discourse and conciliation, and to show how New Zealanders can be effective global citizens. AI is where that is needed now. We all should:

  1. Tell the incentive story to policymakers. Ministers and officials should watch The AI Doc and understand the mechanism of domination, debt and distrust, not just the opportunity.
  2. Back the red line on recursive self-improvement. New Zealand can support it internationally and write it into the terms of any data centre built here, as Harris suggested Iceland could.
  3. Price the risks into sovereign AI. Data sovereignty matters, but every data-centre proposal should be tested against energy, water, diesel, stranded-asset and supply-chain risk, with the community involved.
  4. Turn the wealth pump down. History says societies come through these crises when they reform in time; AI policy should be judged by whether it concentrates or spreads power and value.
  5. Raise the standard of our public debate. Our politicians should model the kind of discussion Iceland’s leaders held in Harpa.

Closing: two imaginaries

In Reykjavík an incredible school choir closed Harris’s AI plenary by singing Imagine, starting in Icelandic and finishing in English. Their rendition was beautiful and inspiring. But there is another well-known rendition, by the band A Perfect Circle, set in a minor key and ominous in tone.

Figure 4. The Icelandic girls’ choir closes the Humanity Dilemma session with Imagine (Matt Boyd, Arctic Circle Assembly, 9 October 2026).

Two versions of the same song. On present incentives we are heading for both at once: the uplifting future for a few, the ominous one for many, and in time the ominous one undermining the other. I left Reykjavík convinced that this is not inevitable. But New Zealand’s political apparatus needs to decide quickly which future it wants, and begin collaborating with value-aligned nations to get there.

Changing the Rules to Soften Humanity’s Hard Landing: A Systemic Risk Approach to Everything Going Wrong at Once

A Tale of Two Conferences Part II: ASRA ‘Currents of Change’ Symposium 2025

(In-depth read, 15 min)

The importance of nature was clear at Les Fontaines – venue for the ASRA Symposium. Photo credit: the author

TLDR/Summary

  • Part II of a two-part blog series reporting on a pair of crisis/disaster risk conferences – this one covers the ASRA ‘Currents of Change’ Symposium, which offered a refreshing contrast to the UN’s symptom-focused approach detailed in Part I.
  • ASRA brought systems thinking to crisis management – 250 multidisciplinary experts tackled interconnected “polycrisis” issues rather than isolated disasters, focusing on the deeper stresses that drive cascading failures.
  • Keynote speakers delivered transformation-focused messages – Poet Ben Okri challenged humanity to become “the people our times require,” while Christiana Figueres emphasised that “linear thinking has no place” in addressing systemic risk.
  • Practical tools emerged alongside theory – ASRA launched STEER, a beta tool for systemic risk assessment, and workshops demonstrated hands-on polycrisis analysis and intervention design using real global stresses and future scenarios.
  • The hard truth: single solutions won’t work – Whether it’s capitalism, carbon emissions, or specific leaders, there’s no single root cause to our interconnected crises; siloed institutions impede the interdisciplinary approaches we desperately need.
  • Bottom line: humanity has the frameworks and community, but the race against time continues – ASRA provided genuine hope and practical starting points, but whether this scales fast enough to prevent humanity’s “hard landing” remains the crucial question.

Definitions

Global systemic stresses: long-term processes that weaken the resilience of critical global systems by increasing pressures, sharpening contradictions, and expanding vulnerabilities. These stresses make systems more vulnerable to trigger events that push them into a crisis.

Polycrisis: The simultaneous occurrence of multiple, interconnected crises that exacerbate each other, creating a situation more severe than the sum of its parts. It’s not just a collection of unrelated crises, but rather a situation where different crises interact and amplify the negative impacts of each other.

Systemic risk: The potential for multiple, increasingly severe, abrupt, differentiated yet interconnected, and potentially long-lasting and complex impacts on coupled natural and human systems. Systemic risk implies the potential for system-level breakdown and cascading consequences across human and natural systems.

Metacrisis: In this blog ‘metacrisis’ refers to the collection of forces: evolutionary, social, technological, and game theoretic, that drive and give rise to global systemic stresses, and resulting crises, polycrisis, and systemic risk.

Introduction & Context

Twenty-four hours after leaving the UN Office for Disaster Risk Reduction Global Platform in Geneva, somewhat pessimistic about humanity’s trajectory, I found myself at Les Fontaines in Chantilly Gouvieaux, France, for an entirely different kind of gathering.

The Accelerator for Systemic Risk Assessment (ASRA) was hosting the ‘Currents of Change’ Symposium – the first transdisciplinary global meeting dedicated to action on systemic risk.

The Symposium offered what the UNDRR Global Platform appeared to miss: clear-eyed analysis of the deeper and interconnected stresses driving cascading crises, coupled with actionable frameworks for addressing them.

ASRA represents a fresh approach to global catastrophic risk. As a network of 90 transdisciplinary experts, it brought together 250 systemic risk practitioners and stakeholders to tackle what ASRA defines as systemic risk: “the potential for multiple, increasingly severe, abrupt, differentiated yet interconnected, and potentially long-lasting and complex impacts on coupled natural and human systems.”

Unlike traditional disaster risk conferences focusing on specific hazards, ASRA addressed the underlying patterns generating cascading failures across interconnected systems. The goal: prevent, mitigate, adapt, and transform away from systemic risk before it overwhelms humanity’s response capacity.

Opening Address: Ben Okri’s Call for Transformation

Ben Okri delivers the opening address at ASRA’s Symposium. Photo credit: the author.

British/Nigerian poet and author Ben Okri gave the opening keynote, a moving, powerful account of humanity’s current predicament that immediately distinguished this gathering from conventional policy conferences. As a renowned novelist, Okri brought a different lens that cut through technocratic language to human realities.

“Many things have come into reality that cannot sustain themselves,” Okri observed – capturing what metacrisis theorist Daniel Schmachtenberger had described as humanity’s “self-terminating race” (see Part I).

But rather than dwelling in despair, Okri challenged humanity towards transformation thinking: “We must not make the mistake of thinking that the present will become the future.”

His diagnosis was unflinching. “Nations cannot talk of making themselves ‘great’ at the expense of making the rest of humanity small,” directly addressing the zero-sum thinking that underlies the competitive dynamics driving many global systemic stresses.

Most crucially: “We cannot combat the difficulty of our times as the people we used to be, we have to be fit and healthy, and we have to create wider and wider communities and alliances and we have to fight the evil of our times intelligently.”

This call for intelligent, collaborative action echoed throughout the Symposium’s technical sessions.

Keynote: Christiana Figueres on Transformative Change

The first keynote session saw Christiana Figueres, former Executive Secretary of the UN Framework Convention on Climate Change and architect of the Paris Agreement, demonstrate how systemic thinking translates into concrete action. Her presentation exemplified the Symposium’s approach of inclusive systematic analysis with practical optimism.

“In the face of systemic risk, linear thinking has no place today,” Figueres began, directly addressing the siloed thinking that limited many of the UNDRR discussions I had attended the previous week (see Part I).

We have the technology and understand interconnectedness, she continued, the question is implementation. Furthermore, rather than aiming to minimise our impact, we should actively seek to restore nature, shifting from ‘sustaining’ to ‘regenerating.’

Costa Rica provided her key example, where laws now facilitate payment for environmental services, resulting in increased forest cover from 29% to 55%. This demonstrates how changing incentive structures drives systemic change.

Figueres used the metaphor of a spider web for interactions in complex systems, explaining that we can’t control the web through top-down decrees, but we can observe “which threads are being pulled and how” and identify effects and leverage points where small changes create large systemic shifts.

Christiana Figueres speaking at the 2025 ASRA Symposium. Photo: the author.

Most importantly, Figueres identified a crucial constraint: “The scarcest resources at the moment are kindness and love.” She warned against letting news feeds crowd out genuine learning sources, including learning from the natural world. In a similar vein, I’ve previously blogged on Jaron Lanier’s calls for deleting all your social media ‘right now’, in my post on Covid, Trump, and algorithms.

Panel Discussions: Scale, Speed, and Systemic Solutions

ASRA Symposium panels tackled how we can meet the scale, scope and speed required for transformation. Unlike conferences focusing on incremental improvements, panellists grappled directly with the need to change human systems.

Participants spoke of bold actions, trust, and “crazy imagination.” One participant noted that, “change happens at the speed of trust,” and “we need to be good ancestors, that’s all.”

But the panellists also honestly assessed barriers, noting for example that Ministers of Finance lack technical understanding of systemic risk. We need bold moves in building systemic resilience so that human systems can handle the stress of the transformation that is required to reduce risk in the long term. However, the current efficiency vs resilience trade-off balance is wrong – we’ve built fragile systems optimised for short-term performance. We must stress-test our systems (whether financial, trade, food, or whatever vital system) and ask if the future we’re creating is resilient to the shocks that are increasingly likely. These stress tests require facilitated dialogue and knowledge sharing across the sectors and systems.

On the required foresight, participants emphasized anticipatory governance as key. Long-term efficiency comes through resilience, not optimisation, because iterated disasters and shocks will undermine efficient systems more over time than resilient systems.

Unfortunately, current crisis response follows whack-a-mole patterns addressing symptoms not causes. Humanity lacks the appropriate anticipatory governance, mechanisms to effect system redesign, and cross-border, regional and global coordination. In particular, we need to stop trying to solve global problems with national tools (as this will lead us into the game theoretic traps and harmful zero-sum dynamics).

“We shouldn’t fix the past, we need to build the future. It was these old systems that have led us here,” noted one panellist. Furthermore, we should act with “good enough information and fast enough action” rather than delays in search of perfect knowledge and optimal decisions.

The way the world seems stuck in rigid historical frames and decision processes, maladaptive in a present world of crisis and existential threats, reminds me of a scene in the film No Country for Old Men. Once he has outwitted the hero, villain Anton Chigurh observes:

“If the rule you followed brought you to this, of what use was the rule?”

The world and its legacy rule-based institutions appear to be in a similar bind. A new system of rules and processes is needed, or humanity will be brought “to this”.

Anton Chigurh, the villain in No Country for Old Men, challenges humanity to reflect on our systems and processes. Miramax Films (2007).

Launch of the STEER Tool: Practical Systemic Risk Assessment & Response

At the Symposium, ASRA launched STEER (Systemic Tool to Explore and Evaluate Risks), a tool designed to help users ‘steer’ away from systemic crisis and toward solutions.

Currently in beta, STEER will make systemic risk assessment accessible to organisations, governments, and researchers across all systems and sectors.

STEER represents practical systems thinking and helps users map interconnections (systemic risk assessment) and identify intervention points (systemic risk responses) for systemic impact, rather than analysing risks in isolation. A combination of drop down menus, tutorial material, and prompts for reflection and action guide users through the processes of systemic risk assessment and response.

STEER will be publicly launched later this year, but feedback from conference attendees  (all experts on systemic risk) beta testing the platform was very positive.

The ASRA Symposium provided opportunities for attendees to engage in the kind of processes that STEER encourages, through various workshops and side-events. I managed to attend two of these.

Workshop 1: Understanding Systemic Risk as Polycrisis

I participated in a breakout session facilitated by the Cascade Institute. This provided a hands-on polycrisis analysis exercise. The workshop highlighted global stresses and groups plotted possible interactions among these along with the triggers that could tip such interactions into crises. This helps us understand why so much is going wrong at the same time.

The theoretical basis for the Cascade Institute’s approach is their stress-trigger-crisis model. The model shows that stresses push systems toward points where triggers might create disequilibrium (and likely associated harm in human and ecological systems). Even without triggers, inexorable stressors will push systems into potentially harmful new states (eg, as the left hand depression in the figure below becomes shallower). Averting crises requires acting on stresses of three types: pressures, contradictions, and vulnerabilities.

Figure credit: Cascade Institute’s ‘Stress-Trigger-Crisis’ model (2024)

The Institute previously identified 14 global systemic stresses which create cascading failure conditions for humanity, and which must be addressed to have hope of mitigating the present polycrisis (you can read more about these here):

  • Climate heating
  • Ecological degradation
  • Toxicity
  • Zoonotic disease transfer
  • Demographic divergence
  • Concentrated industrial food production
  • Changing energy supply
  • Financial interconnectedness
  • Economic headwinds
  • Economic inequality
  • Ideological fragmentation and polarization
  • Political-institutional decay
  • Great power hegemonic transition
  • Propagation of artificial intelligence

Working groups mapped interactions between three assigned stressors each, analysing how crises emerge when triggers act within these interactions. Each crisis can become a trigger within other patterns.

The messy but structured and necessary process to understand complex systemic risk. Photo credit: the author.

For example, my group was tasked with considering interactions among:

  • North-South demographic divergence
  • The concentrated nature of industrial food production
  • Rising economic inequality

Interactions between these factors could be stressed further by events such as a policy shift in migration settings, or synchronous heatwaves in critical food production regions, leading to a crisis of workforce availability and food production, resulting in famine or war, with these crises then being the triggers of other global crises in cascading fashion.

The exercise rapidly demonstrated how current conditions create multiple, interacting, cascading failures – a polycrisis rather than isolated events. And we only considered three of the 14 global stresses!

Crucially, we brainstormed interventions for crisis mitigation through anticipatory action, such as sensible migration policies, sustainability criteria on imports, more heterogeneous distributed food systems, with food system buffers, and policies that alleviate economic inequality to hedge against short-term price shocks.

Key insight: There’s no single root cause of a polycrisis. It is not simply capitalism, carbon emissions, or the actions of particular leaders, but everything in conjunction. Single-point solutions won’t work. Siloed institutions impede solutions, which require interdisciplinary complex systems thinking.

Workshop 2: Preparing for Catastrophic System Failure

Another workshop facilitated by David Korowicz addressed whether catastrophic system failure can be mitigated ahead of time. We contemplated a scenario where (for the purposes of the foresight exercise) a national Cabinet has knowledge that a catastrophe severely decreasing goods, services, and energy access will happen in either 1, 4, or 8 years. Our group was tasked with considering how we would act with such information under the 4-year time horizon.

Roughly the results of our deliberations can be summarised as follows:

  • Prevent panic while being clear resilience is a fundamental priority and the nation needs to start seriously working to mitigate likely effects of future crises.
  • Assess physical security and available resources, ensuring physical safety and liaising with trusted international partners.
  • Strengthen connections at all levels across government and society (families, communities, regions, international).
  • Analyse complex reactions to crisis – how will people and countries respond? Will there be national hoarding with export controls? Ensure appropriate engagement with behavioural scientists.
  • Map consequences for energy, transport, food, and communications systems.
  • Stocktake the minimum functions required to sustain society according to hierarchy of needs (water, food, shelter, energy, etc).
  • Develop mitigation options for each critical function in context of the catastrophe.
  • Ensure redundant structures for communications, food, shelter.
  • Strategic stockpiling while understanding supply constraints from other jurisdictions doing the same.
  • Roll out incentives for electrification, local biofuels, distributed food production, and other resilience measures.
  • Sequence and prioritise all interventions for maximum effectiveness.

Admittedly all the above were developed on the fly in half an hour, but the exercise raised two key questions for me.

  • First, this all sounded incredibly familiar, and is basically the content of our own organisation’s detailed report on New Zealand’s vulnerability and resilience options against the risk of Northern Hemisphere nuclear war.

Recent media reporting on our study can be found here. You can read the rich and detailed report here, which is effectively a maturity model for resilience to global catastrophic risks, including one-page ‘cheat sheets’ for each key sector and for global catastrophic risk management.

  • Second, why haven’t governments of the world conducted this kind of exercise, and developed and implemented exactly these plans and programmes, in conjunction with their citizens, given the perilous state of the world?

The workshop discussions also highlighted that this kind of resilience doesn’t depend on nationalistic self-sufficiency but on creating systems that are less susceptible to cascading collapses: locally resilient food and energy, regional governance, delinking from fragile global finance, mutual support networks. But also, and importantly, linkages with regional partners, collaborations of nations to ensure trade and supply through investment in strategic infrastructure and plans, and the avoidance of hoarding, which although seems rational for individual jurisdictions, could actually lower the global mean ability to ride out the crisis, creating overall more harm.

Addressing the Causes of Global Systemic Stresses themselves

The workshops demonstrated that there are positive steps humanity can take towards limiting future catastrophe harm, even harm stemming from system-wide failures. We can implement systems thinking, map systemic interactions, develop resilience through anticipatory governance, conduct preparedness exercises, and reduce the human and environmental impact, and therefore depth of the economic harm that global systemic risk threatens.

ASRA’s greatest contribution was acknowledging this challenge while providing concrete intervention tools. Much more work is needed, particularly to address what drives these global stresses, including rivalrous dynamics preventing coordination, exponential technological advancement creating risks faster than assessment is possible, and resource degradation amid coordination failures. The impact of global stress reduction through systems thinking and action may still not be enough, because civilisation’s underlying dynamics don’t support such action. We’re potentially stuck in evolutionarily stable strategies where aggressive, exploitative behaviours outcompete cooperative, long-term alternatives – even when cooperation ensures collective survival.

All that said, the frameworks discussed and exercised at the ASRA Symposium offer genuine starting points for a new cognitive frame and for systemic intervention.

Conclusion: Building on Systemic Foundations

The contrast between the UNDRR Global Platform and the ASRA Symposium was striking and illuminates the limitations of current disaster risk reduction efforts, which are probably largely driven by historic silos.  

While UNDRR demonstrated genuine commitment to developing resilience, discussion remained trapped within frameworks addressing symptoms rather than systems. ASRA took a fundamentally different approach, placing difficult systemic questions at the analytic heart. The result was honest assessment combined with practical intervention tools.

Most importantly, ASRA provided a transformation-focused community of practice. The Symposium demonstrated that systemic risk assessment isn’t an abstract academic exercise. It’s an urgent practical necessity for decision-makers navigating interconnected worlds where risks cascade faster than traditional approaches can address. This practical necessity needs to be resourced. Analysts and decision-makers across all vital sectors and systems need time and space to cooperate, coordinate, and hash-out these problems around the same table.

The challenge of disaster risk reduction, building immunity to global catastrophic risks, and transforming human systems away from those that generate these risks remains enormous. Changing competitive dynamics and evolutionarily stable strategies requires changes from individual consciousness right up to global governance.

We can respond and recover from various crises, we can build resilience and mitigate the impact of future crises, we can reduce systemic risk through judicious systems transformation, we can mitigate the polycrisis by minimising the global systemic stresses, but only by intervening on the forces comprising the metacrisis can we prevent global stresses and crises being thrown up again and again, in increasingly severe form.

I reported Daniel Schmachtenberger’s views in Part I. He notes that the race dynamics of humanity are self-terminating. Individual improvement is insufficient – we need to bend the entire arc of human history. Ben Okri echoed this at ASRA: “We have to find better alternatives to the current direction of history.”

But frameworks, tools, and community emerging from initiatives like ASRA provide hopeful foundations and Ben Okri’s challenge echoes as warning and invitation. We cannot combat our times’ difficulties as the people we used to be, but we can choose to become the people our times require.

Whether this mindset and process scales and accelerates quickly enough to bend the arc of human history before the “hard landing” becomes inevitable remains the question.

Lessons from the Cambridge Conference on Catastrophic Risk 2024

Video: Dr Matt Boyd presents highlights of NZ’s vulnerability and resilience to nuclear war and other global catastrophes. You can download a PDF version of this presentation to access all links.

Blog Summary/TLDR

  • Diverse researchers, analysts, and officials gathered at the University of Cambridge’s Centre for the Study of Existential Risk (CSER) in September 2024 to discuss global catastrophic risks and potential solutions.
  • Key topics included: emerging risks, the systemic nature of risk, resilience options, comprehensive risk governance, and partnerships for change.
  • The conference highlighted the expanding and interdependent nature of global risks, the challenges in managing them, and the need for better international cooperation and coordination in risk assessment and governance.
  • Practical solutions included: inclusion of global risks in national risk assessments, new international agreements, national chief risk officers, and on-the-ground solutions to the local manifestations of catastrophe.
  • New Zealand needs to improve its preparedness for global catastrophe and consider measures such as ensuring local fuel supply, upgrading critical infrastructure, and developing a publicly facing National Risk Register that includes global catastrophic risks.
  • Click the YouTube video above to watch Dr Matt Boyd’s conference presentation.
  • For more on these issues, you can read the NZCat Main Report about NZ’s vulnerability and resilience to nuclear war or visit Islands for the Future of Humanity.

The CSER Conference

On the 17–18th Sept 2024 I joined a diverse set of researchers and analysts converging on the University of Cambridge’s Centre for the Study of Existential Risk (CSER) to contemplate risks that could result in the collapse of civilisation.

Alongside catastrophic risk researchers and analysts from the global North and South, attendees included diplomats, representatives from the UN, think tanks, governments such as Singapore, NATO advisors, and students.

Tongue-in-cheek dinner speaker Lance Gharavi professed the merits of the comical Centre for Applied Eschatology, however most in attendance had the goal of preventing global catastrophes such as nuclear war, extreme pandemics or technological disasters, and ensuring recovery should these catastrophes ever befall us.

Keynote speaker Mami Mizutori, a former Special Representative of the UN Secretary-General for Disaster Risk Reduction, talked about four needs in the face of potential global catastrophe:

  • Comprehensive risk governance
  • Partnerships for change
  • Better synergy among global agendas
  • Courage to tackle systemic risk

But Mizutori also warned that there is a difficult path ahead with multilateralism and binding agreements being difficult to achieve in 2024.

Peter Sogaard Jorgensen spoke about the evolution of the polycrisis we now face, and the structural drivers of risk that have landed the world amid a set of ‘anthropocene traps’. Understanding the evolution of these structural incentives is key to overcoming them.

The conference heard grass-roots solutions to major risks including flood resilience in Pakistan (from Sabuhi Essa), and the importance of indigenous knowledge and rights (Elena Kavanagh). We also heard about ‘derailment risks’ (Laurie Laybourn) where solutions to catastrophes such as climate change could become unattainable as our capability to solve them is undermined by the very catastrophe process itself.

Emerging risks discussed included biological and other converging technological threats (Margaret Kosal), the risk of space wars and satellite disruption (Joanna Rozpedowski), including the blurred lines between what is a military target vs civilian asset. 

The audience heard new approaches to risk analysis and mitigation such as ‘impact webs’ (Edward Sparkes) or ‘webs of prevention’ (Catherine Rhodes), as well as the value of a potential UN convention on existential risk (Manfred Kohler).

A session on resilience to global catastrophe highlighted solutions such as fuel security for agriculture in NZ (this author: Matt Boyd), resilient foods (Juan Garcia), and the need for government plans. One success reported by Monica Ulloa of the Observatorio de Riesgos Catastróficos Globales, was the government of Argentina including abrupt sunlight reducing scenarios (such as volcanic or nuclear winter) in government risk analysis. Argentina is the first country to do this.

I had informative and stimulating conversations ranging from the importance of government Chief Risk Officers to coordinate anticipatory risk governance, through to first-hand accounts about wealthy individuals looking to identify resilient locations to build secure refuges.

It’s impossible to give full details of the 50 speakers and dozens of poster presentations here, but the overall picture was one of expanding major global risks (due to rising complexity, interconnectedness and human impacts) some which may be beginning to elude our capacity to manage them, and an international system not up to the task of coordinated risk governance.

But solutions are possible, and takeaways included the urgent need for governments to add global catastrophic risks to risk assessment and risk management processes, and perhaps more critically, to cooperate and coordinate on cross-border risks.

We have recently seen the UN Member States adopt a Pact for the Future, which explicitly calls out global catastrophic and existential risks and addresses climate, nuclear, biological, and technological risks. These ambitious statements must now be backed through action of Member States individually and collectively. 

I also had the opportunity to take part in two of Lara Mani’s (CSER) global risk workshops.

The first was a creative workshop focused on practical creation and dissemination of risk information and key messages in succinct and accessible forms.

In the second, I played the role of UK Minister for Health in a scenario-based workshop contemplating firstly an Indonesian supervolcano eruption, the cascading consequences of which spread to affect the entire world, and secondly, a scenario dealing with the catastrophic collapse of the UK power grid.

These exercises in communication and understanding of catastrophic risk were very effective and governments should undertake such exercises regularly.

Not all risk communication should be dry research reports: Communication workshop facilitated by Lara Mani (CSER) – Author’s photo

Reflections

On reflection after the conference, I can commend the government of my own country (NZ) for some of its recent initiatives:

  • National Fuel Security Study – to investigate the options for ensuring liquid fuel supply, however this study needs to contemplate global catastrophic risks explicitly.
  • Draft Critical Minerals List – for public consultation to inform strategic development of essential mineral resources (see our submission that takes a global catastrophe perspective here).
  • Draft Resolution on Nuclear War Effects and Scientific Research, announced with Ireland, that proposes an up-to-date international study on the effects of nuclear war (ideally including second order cascading effects).
  • National Emergency Management Agency ‘CatPlan’ handbook for hazard agnostic catastrophe management – not released yet. It appears this handbook will discuss catastrophes ‘requiring international support’, however some scenarios need to assume such support is not forthcoming.
  • Given the last point, NZ needs to assess its capital stocks and ensure that the available human, natural, physical, and financial capitals are sufficient to provide resilience options in the radically altered context of a true global catastrophe.

As such, the NZ Government can still learn from global examples:

  • Argentina’s inclusion of abrupt sunlight reduction scenarios in national risk assessment (ie, nuclear, volcanic, or comet/asteroid winters).
  • The Singapore Government’s attendance at the Cambridge Conference on Catastrophic Risk as part of its ongoing foresight capabilities.
  • The United Kingdom and other countries’ publicly facing National Risk Registers.
  • The United States Global Catastrophic Risk Management Act, which mandated a report on these risks that is due imminently.
  • The many global catastrophe scenario exercises being undertaken around the world to understand the needed preparation and management for risks such as: extreme pandemics, nuclear war/winter, supervolcano, asteroid/comet impact, catastrophic electrical grid or communications failure, or catastrophic global food shortage.

NZ needs to deploy a systemic approach to vulnerability and resilience using a global catastrophic risk lens. Key vulnerabilities in NZ include single failure points that must be mitigated as a priority. This includes (among other initiatives):

  • Ensuring local supply and production of liquid fuels (eg, biofuels) for critical processes such as agriculture, should fuel imports cease.
  • Ensuring reliable and future-proofed Cook Strait ferries, which are currently so critical given NZ remains over-dependent on road truck freight for distribution of food and essential goods.
  • Upgrading coastal shipping and rail assets and associated infrastructure to facilitate trade, including with Australia, in the context of a dire global catastrophe.
  • Developing a Digital Communications Continuity Plan.

NZ could develop and leverage a publicly facing National Risk Register connected to a set of solution visions, combined with the Infrastructure Commission’s Infrastructure Priorities Programme to ensure the required resilience is developed. I encourage people to submit Stage 1 proposals to the Commission highlighting key national vulnerabilities.

We have prepared a range of materials the NZ Government and other organisations can use in this mission, and interested readers can visit: