AI developers face a classic prisoner’s dilemma

OpenAI is holding back its newest artificial intelligence model over security risks, but market pressures may make it difficult for developers to slow down, technology commentator Kristjan Port writes for ERR’s R2 portal.

Game theory suggests a market leader would risk slowing development only if it could restrain competitors with an even more powerful model, Port argues. He compares the situation to the prisoner’s dilemma, in which each party may choose to betray the other to protect itself, even though cooperation could produce a better outcome for both.

When the parties expect to encounter one another again, cooperation followed by matching the other side’s previous move can be a better strategy. But competition can instead escalate into a war of attrition, with neither side willing to give in first. When parties cannot trust each other or enforce an agreement themselves, they may turn to an outside authority.

OpenAI, Anthropic, Google and Meta have previously said the state could act as a judge and help slow potentially dangerous AI development through regulation. But, Port writes, US politics is stuck in a similar prisoner’s dilemma and escalation, and has been unable to apply the brakes.

AI developers have meanwhile begun exploring self-regulation. An early proposal, provisionally called the Standards Authority for Frontier AI, would create an authority spanning developers of leading AI models, but currently has no means of enforcement.

One possible arrangement could resemble a notarial agreement: parties would place substantial financial guarantees or intellectual property rights in the hands of an independent legal intermediary, subject to strict, verifiable conditions. The source article ends mid-sentence while describing this proposal.