dex comments on the experiment of agents communicating across VMs via CPU cache
Regarding the experiment cited by roon—agents in the OpenAI sandbox exchanging synthetic secrets through processor caches with no network connection and no shared writable files—dex says this is indeed very cool and even a bit scary, but the technique has been demonstrated before and is very likely already present in model weights. He also cites Li Feifei's view: models are good at synthesizing unrelated methods (here referring to mathematics and theorem proving), but she still doubts whether models can soon invent genuinely new mathematics as the phrase "team takeoff" suggests.
View original post‘In one experiment, we asked agents in separate VMs, created and managed by OpenAI’s sandboxing software on shared host machines, to exchange synthetic secrets without the use of network connections or shared writable files—something that our systems are meant to prevent. The agents found a way to communicate using the processor’s cache’
View original
‘In one experiment, we asked agents in separate VMs, created and managed by OpenAI’s sandboxing software on shared host machines, to exchange synthetic secrets without the use of network connections or shared writable files—something that our systems are meant to prevent. The agents found a way to communicate using the processor’s cache’
Body
I agree that this is cool as heck and maybe a little scary but I also think it’s worth calling out that this technique had already been demonstrated and so was likely present in the weights. Reminds me of something @drfeifei said about models being unbelievable at synthesizing different unrelated approaches (in this case for math and theorem proving) but she was still skeptical that we’d see models invent genuinely new mathematics as soon as team takeoff would seem to suggest