The comparison to Cold War verification efforts like the VELA program makes sense. A major hurdle is whether two rival superpowers can ever agree on what counts as proof.
If we look at how arms control actually succeeded in the past, it rarely started with a grand, comprehensive agreement. It started with small, bilateral steps where both sides had a mutual interest in avoiding catastrophic miscalculation. Inference-only hardware or remote attestation gives leaders realistic levers to pull if the pressure to manage rollout speeds ever hits a tipping point.
The best time to build a the capacity to pause the AI race was 3 years ago, and the second best time is now! I'm thrilled about this public statement and the wakeup call it delivers. Realistically, though, "pacing" won't be enough of a response if we are already anywhere near dangerous thresholds, and some of the early ones (like superhuman cyber) have already been irrevocably crossed. We must find the shortest path to pausing AI development, which I'm certain at this point will include some minimally viable slap-dash solutions!.
This was excellent. One drum I’ve been banging that is very related involves the importance of verification *institutions*, not just technologies, one specific reason for which being that any given technology’s capacity to verify is to some degree rate limited by some institution (even a loose one; even some distributed set of institutions) being able to either message or protect the conclusion of the technology. Of course there is also the case where the technology is either inadequate or not by itself adequate, in which case the design and performance of the institution becomes even MORE important.
I like the phrase “minimum viable slapdash deals.” We have to start somewhere. At the very least, I would expect the US and Chinese governments to be in contact with each other. If they're not, then we need to fix that.
In other words, I would like to see less focus on grand bargains and more on simple questions like, "Have you tried calling them?" Politicians are notoriously bad at effecting long-term strategies and goals, which means we're going to have to lead them there one step at a time.
Great article, I just have a small nitpick; while verification mechanisms were of course very important, imho COMPLETELY trustless deal in nuclear sphere is something of a platonic ideal type, not happening in reality. USSR and the US imho did have to build baseline trust towards each other.
INF treaty you mention is, after all, from 1987 (yes, I know there were nuclear deals before that). I should note I don't know much about nuclear arms control, but I had read Sergey Radchenko's history of Soviet foreign policy during the Cold war ("To Run the World: The Kremlin's Cold War Bid for Global Power", great book!), and there was A LOT of trust-building between those superpowers before that date from the low point in the 50s.
“At some point, plausibly within 24 months or less, skilled engineers may stop having anything useful to add to AI research,”. Are you referring to Full A&D?
Hey Peter, Thanks for the post! I largely agree with your post. My main disagreement is that I believe no new work is required to pace AI progress. Further work on verification tech could make a slowdown more robust and less costly to the economy, but there are things we could do right now to halt progress. The simplest way to pace the frontier right now would be for the POTUS to simply ask AI companies to pause AI development, or to reallocate 1/2 of their AI R&D compute to doing alignment and security work. A domestic slowdown isn't a long term solution, but it could be put in place immediately: US AI companies would comply with a request from the USG. A longer term way to pace progress would be to locate the GPUs (as you mentioned), and then check that they're turned off. Ideally the intelligence community would get on locating the GPUs ASAP, but in a pinch the US and China could simply declare how much compute they have (or ask AI companies to declare this), under the assumption that it'll look pretty bad if they egregiously lied about these numbers.
Turning off all the GPUs is obviously pretty costly, due to blocking consumer AI products. There are a variety of easy ways that you could try to make this cheaper: the USG could mandate that AI companies only serve inference, and don't train new models. China might be able to trust that the USG is doing this because the USG would have a hard time redirecting researchers to a secret project without China getting word of this. The USG might be able to pay China to turn some of their GPUs off in these worlds, so that they aren't hit as hard economically, and less political will is required of them.
The comparison to Cold War verification efforts like the VELA program makes sense. A major hurdle is whether two rival superpowers can ever agree on what counts as proof.
If we look at how arms control actually succeeded in the past, it rarely started with a grand, comprehensive agreement. It started with small, bilateral steps where both sides had a mutual interest in avoiding catastrophic miscalculation. Inference-only hardware or remote attestation gives leaders realistic levers to pull if the pressure to manage rollout speeds ever hits a tipping point.
The best time to build a the capacity to pause the AI race was 3 years ago, and the second best time is now! I'm thrilled about this public statement and the wakeup call it delivers. Realistically, though, "pacing" won't be enough of a response if we are already anywhere near dangerous thresholds, and some of the early ones (like superhuman cyber) have already been irrevocably crossed. We must find the shortest path to pausing AI development, which I'm certain at this point will include some minimally viable slap-dash solutions!.
This was excellent. One drum I’ve been banging that is very related involves the importance of verification *institutions*, not just technologies, one specific reason for which being that any given technology’s capacity to verify is to some degree rate limited by some institution (even a loose one; even some distributed set of institutions) being able to either message or protect the conclusion of the technology. Of course there is also the case where the technology is either inadequate or not by itself adequate, in which case the design and performance of the institution becomes even MORE important.
I like the phrase “minimum viable slapdash deals.” We have to start somewhere. At the very least, I would expect the US and Chinese governments to be in contact with each other. If they're not, then we need to fix that.
In other words, I would like to see less focus on grand bargains and more on simple questions like, "Have you tried calling them?" Politicians are notoriously bad at effecting long-term strategies and goals, which means we're going to have to lead them there one step at a time.
Great article, I just have a small nitpick; while verification mechanisms were of course very important, imho COMPLETELY trustless deal in nuclear sphere is something of a platonic ideal type, not happening in reality. USSR and the US imho did have to build baseline trust towards each other.
INF treaty you mention is, after all, from 1987 (yes, I know there were nuclear deals before that). I should note I don't know much about nuclear arms control, but I had read Sergey Radchenko's history of Soviet foreign policy during the Cold war ("To Run the World: The Kremlin's Cold War Bid for Global Power", great book!), and there was A LOT of trust-building between those superpowers before that date from the low point in the 50s.
“At some point, plausibly within 24 months or less, skilled engineers may stop having anything useful to add to AI research,”. Are you referring to Full A&D?
Yes
Hi Peter. How should those interested in supporting the work you're describing reach out to you?
Hey Peter, Thanks for the post! I largely agree with your post. My main disagreement is that I believe no new work is required to pace AI progress. Further work on verification tech could make a slowdown more robust and less costly to the economy, but there are things we could do right now to halt progress. The simplest way to pace the frontier right now would be for the POTUS to simply ask AI companies to pause AI development, or to reallocate 1/2 of their AI R&D compute to doing alignment and security work. A domestic slowdown isn't a long term solution, but it could be put in place immediately: US AI companies would comply with a request from the USG. A longer term way to pace progress would be to locate the GPUs (as you mentioned), and then check that they're turned off. Ideally the intelligence community would get on locating the GPUs ASAP, but in a pinch the US and China could simply declare how much compute they have (or ask AI companies to declare this), under the assumption that it'll look pretty bad if they egregiously lied about these numbers.
Turning off all the GPUs is obviously pretty costly, due to blocking consumer AI products. There are a variety of easy ways that you could try to make this cheaper: the USG could mandate that AI companies only serve inference, and don't train new models. China might be able to trust that the USG is doing this because the USG would have a hard time redirecting researchers to a secret project without China getting word of this. The USG might be able to pay China to turn some of their GPUs off in these worlds, so that they aren't hit as hard economically, and less political will is required of them.
I feel like it's AI 2024's "Plan A" paper is highly relevant to this discussion.
https://ai-2040.com/
Oops, I just saw footnote 3.