Discussion about this post

User's avatar
Herbie Bradley's avatar

I think this is a bit overstated:

- Mythos cyberoffense capability probably does not rival that of nation states (who actually invest in maintaining a cyberoffense team). My mental model for a while has been that AI's comparative advantage in cyber offense is in scaling attacks that do not depend on high degree reliability or coordination across multiple attack vectors—i.e., ransomware attacks, not top tier nation state hacks. SL3 in RAND parlance.

- you seem to be doing the "generalize from one capability to many others" mental motion, but I think a fairly standard "capabilities are jagged" + "depending on easy to verify/hard to verify tasks" model would predict both the great gains in cyber + that models will improve much more slowly at eg strategy.

- on superintelligence, I'm starting to desire this definition to be broken down more, since the further we progress the more it suffers from the same issues as AGI. Currently it can be basically used to imply any level of capabilities desired, even those dependent on accumulating vast resources.

- "Overnight, every intelligence community operation that depends on signals exploitation is potentially compromised." seems vastly overstated? I think it would have broadly been fine, if a bit bumpy, if Mythos-level models are public.

- Agree on the need for govt preparation and to prevent model weight theft!

Connor Heaton's avatar

Excellent thoughts!

Something I'm not hearing discussed much is what state actors are doing with their current zero-day stockpiles.

I would expect that as soon as they saw the exploit stats, and that glasswing was too large a footprint of people to persuade to leave their zero-days unpatched, they would start using them to achieve goals, even if inefficient, as quickly as possible.

Presumably, every day more of the vulnerabilities identified by mythos are being patched, and nation states have little way of knowing if mythos will uncover an exploit they paid 800k for, so it's quickly a use it or lose it situation. I'm sure some will be kept in reserve, but it's a gamble. So stockpiles are likely being deployed now in ways and for goals which potentially won't become public for many years.

The NSA must very upset about mythos, especially given the WH-anthropic feud. Mythos also reduces the extent to which the NSA can maintain a capabilities lead through recruiting top cyber talent.

32 more comments...

No posts

Ready for more?