Lots of great stuff here - the load-bearing ness of empirical problem definition to RSI; the lack of commercialization incentive to deploy internal S-tier tools publicly (in stark contrast to eg Amazon). But imo I’m most compelled by the imperative towards third-party verification institutions, which themselves I think will have to be RSI enabled. How IYO do you think we can crack the nut on the internal talent needed to execute against that brief sustainably? Do you think the will towards contribution along that axis is sufficient to overcome potential conflicts of interest (eg ‘donated time’ from labs?)
Thanks! I do think this is a concern. IIUC — the talent concern is that the AI companies pay vastly more than government? (ie. 500k+ vs. <100k)
At this point I think the leading AI companies have pretty good will, even up to Sam Altman / Dario Amodei.. but their relationships may become increasingly adversarial with the government as they become larger ..
One key point might be ensuring government access to (1) models, including internal deployments, (2) logs of what those models have done (e.g. chat history, agent rollouts, etc. things that the companies already keep) — this might be a bigger bottleneck than oversight capacity itself!
Right, yeah, exactly - like it's precisely the equilibrial dynamic changing that is the thing to hedge against. But I do think that a lot of what can help is the kind of stuff you are talking about here.
The widening gap between what researchers are producing in AI company settings and what the public understands is a significant vulnerability.
Lots of great stuff here - the load-bearing ness of empirical problem definition to RSI; the lack of commercialization incentive to deploy internal S-tier tools publicly (in stark contrast to eg Amazon). But imo I’m most compelled by the imperative towards third-party verification institutions, which themselves I think will have to be RSI enabled. How IYO do you think we can crack the nut on the internal talent needed to execute against that brief sustainably? Do you think the will towards contribution along that axis is sufficient to overcome potential conflicts of interest (eg ‘donated time’ from labs?)
Thanks! I do think this is a concern. IIUC — the talent concern is that the AI companies pay vastly more than government? (ie. 500k+ vs. <100k)
At this point I think the leading AI companies have pretty good will, even up to Sam Altman / Dario Amodei.. but their relationships may become increasingly adversarial with the government as they become larger ..
One key point might be ensuring government access to (1) models, including internal deployments, (2) logs of what those models have done (e.g. chat history, agent rollouts, etc. things that the companies already keep) — this might be a bigger bottleneck than oversight capacity itself!
Right, yeah, exactly - like it's precisely the equilibrial dynamic changing that is the thing to hedge against. But I do think that a lot of what can help is the kind of stuff you are talking about here.