Holding the Tiger by the Tail

In case you missed it, on September 12, the CEOs of both OpenAI and Anthropic called for a slowdown in AI development to avoid producing technology that humans cannot control. Their driving concern is to reduce the potential for catastrophic effects AI might unleash on humanity.

Avoiding catastrophe certainly sounds like a good idea. And the idea of a slowdown sounds like prudence. But while a slowdown addresses the pace at which new capabilities might develop, we are still left with the dangers of the capabilities already in place.

This news conjures a familiar kind of cultural trope, one in which runaway technology escapes containment and wreaks havoc on the world. It is a theme as old as Mary Shelley’s Frankenstein and has been echoed repeatedly over the years as in Czech writer Karel Čapek’s 1920 play R.U.R. (which introduced the word robot) and in the 2004 movie I, Robot. And of course, we mustn’t forget other notable movies like The Terminator and 2001: A Space Odyssey, which introduced their own variations on the theme. There are many more examples.

The weight of these cultural works leaves many people with a clear picture of the present danger. A danger that prompted the warning from the two CEOs of the most advanced AI companies in the world.

Unfortunately, that picture is largely wrong.

Or maybe I should say incomplete. Because the potential for AI-induced catastrophe isn’t restricted to the single scenario of runaway technology. There are multiple unique ways in which AI might do massive damage, falling into these seven broad scenarios:

1. Biological/Chemical Weapons Development
2. Cyberattacks
3. Accidental Military Escalation
4. Information-System Failure
5. Rapid Economic Displacement
6. Unreliable Control of Critical Functions
7. Runaway Loss of Control

You can find a detailed chart of the scenarios, mitigation strategies, and the nuances here: AI Catastrophe Scenarios and Mitigation Strategies.

Only scenario number 7 falls within the familiar Frankensteinian trope. But all of them possess the potential for great human harm.

One gut response might be to use the Frankenstein story as inspiration, to storm the castle and put things back the way they were.

We can see a version of that response in the current discussions calling for an AI “kill switch”. This idea is to require the providers of advanced AI models to maintain the ability to quickly shut them down.

It’s an interesting idea. Maybe even a good idea, but that approach also has a problem.

The truth is that each of the seven scenarios has its own unique mitigation strategy. And the real kicker is that four of the seven mitigation strategies depend on AI, and two more are helped by it. In summary:

1. Biological/Chemical Weapons Development: AI will be useful to screen research queries and to evaluate models for dangerous biological knowledge before release.
2. Cyberattacks: AI will be essential to detect intrusions and discover new vulnerabilities.
3. Accidental Military Escalation: This is the only scenario where it might simply be too dangerous to use AI in mitigation. It is essential that humans remain in the loop and in control.
4. Information-System Failure: AI will be essential in detection of deepfakes, impersonation, and coordinated campaigns.
5. Rapid Economic Displacement: AI can have a helpful role in overcoming economic displacement by helping match skills to jobs and designing personalized retraining.
6. Unreliable Control of Critical Functions: AI will be essential to monitor other AI systems, detect abnormal behavior, and avoid cascading failures.
7. Runaway Loss of Control: AI will be indispensable in behavioral monitoring and detecting attempts to bypass safeguards.

These mitigation strategies at times have competing needs. For example, having an AI kill switch may indeed be a good solution to avoid the seventh scenario of Runaway Loss of Control, but it would need very careful implementation to prevent derailing the ongoing mitigation efforts for the other AI disaster scenarios. A case in point: Cyberattacks. Pressing the kill switch to prevent an impending loss-of-control situation might have the unintended side effect of enabling cyberattacks elsewhere, since the cyberattack defenses might be running on the very same models the switch would shut down.

Perhaps less obviously, since AI will be monitoring for other loss-of-control situations, the kill switch might leave us blind to incidents beyond its reach: foreign models, open-weight models, and agents already on the loose.

These are just two examples of how these efforts can become entangled.

There is not universal agreement on this topic. Some vocal in this discussion have discounted both the dangers and the urgency. President Trump labeled AI fears a “hoax” and Nvidia’s Jensen Huang called them overblown “doomsday narratives”. On the other hand, Nobel laureate Geoffrey Hinton, often referred to as “The Godfather of AI”, recently stated Congress has “maybe a year, but not much more than a year” to act and has joined the calls for a slowdown in development.

Meanwhile, the US House has adjourned until after the fall elections and a recent Senate attempt at a “kill switch” bill was blocked. The situation is reminiscent of the fictional American President in the movie Don’t Look Up. After being confronted with the imminent threat of destruction, the decision was simply to “sit tight and assess”. It seems sometimes life imitates art.

At this juncture, we, as a species, are now faced with an enormous problem that is both urgent and complex. On the one hand, our time to act is limited. And on the other hand, simple solutions are liable to do more harm than good. The uncomfortable truth is that, whether we like it or not, we will need to use AI to counter the potential negative effects of AI.

Which brings up a different image: that of a man holding a tiger by the tail. It isn’t safe to hold on or to let go.

And there we are.

What can an individual do? To continue with the theme of using AI to help us manage AI, I asked Claude for suggestions. It seems that each of the seven scenarios carries its own unique action recommendations. Four scenarios allow for direct leverage, where we, as individuals, can take immediate steps. The other three scenarios rely on political efforts where we will need to help our leadership make wise and informed policy decisions. All are summarized on the reference chart: AI Catastrophe Scenarios and Mitigation Strategies.

But there is one recommendation that spans them all:

Evaluate any proposed AI law by asking which of the seven scenarios it addresses and whether it damages defenses against the others.

We are holding the tiger by the tail. And since we can’t let go, the only course is to hold on, and hold on well.