Import AI 469: Science AI, RSI Simulators, and Zuckerberg's Technological Pessimism

DiG-bench shows that Fable displays some creative intuition: …The new frontier for analyzing AI systems is understanding how good they are at inferring the unwritten rules of their environment… How well can AI systems figure out the rules of their environment through exploration and curiosity, versus being fed them? That’s an important question for better understanding the intuitive and creative capabilities of AI systems and it’s one being asked by DiG-bench (Discovery in Games), a new benchmark of 70 games “designed to map the surface of discovery in well-controlled interactive systems”. Similar to the visual ‘ARC’ game, in DiG-bench “each game is a self-contained miniature world with its own laws, but both the rules and the objective are hidden from the player and must be uncovered through interaction”. You can play some of the games yourself online to get a feel for them at the official project website ( digbench.ai ). The key thing this is measuring is the ability for players to spot the important mechanics that determine their success - basically, by playing around with the games you get a sense for how your actions change the environment and through this you also uncover mechanics that you must understand to succeed at the game. The idea is that if you can solve these games you have a decent ability to spot important information in novel environments and update your priors. Who did the research: The authors come from Thinking About Thinking, University of Oxford, Princeton University, King Abdullah University of Science and Technology, Swiss AI Lab, Inria, MIT. One of the authors is Juergen Schmidhuber, an extremely creative OG AI researcher. Key facts:

Purely text-based: The games are basically native to language models. They are also mostly “short enough that most traces fit entirely within the context window of current frontier models”.

Handcrafted and novel and private: All of these games have been built by human experts. The majority of the games are kept private so that AI systems don’t train on them.

Beatable but difficult: Every game has been beaten by at least one human “but players reported finding many games difficult”.

Varied skills: Solving all these games requires different skills and strategies.

Experimentation : The games come with an optional experimentation mode which lets people play around with them without having as intense a “step limit” on actions they can take.

Reassuringly hard: The games are difficult enough that they are not beatable by today’s frontier models.

How well do AI systems do? The benchmark is split into seven tiers with tier 1 being the easiest and tier 7 the hardest. 21 games have been released publicly with the remaining held back. Most of the games have multiple levels and the number of available actions for players to take at each step ranges from 2 all the way up to 34.

Opus 5 and Fable 5 with Claude Code are the best overall models, followed by GPT-5.5

Only Opus 5 and Fable 5 were able to beat any tasks (0.2) in (Tier 7). Opus 5, GPT-5.5, and Kimi K3 were able to beat some tasks in Tier 6 when given access to a harness (e.g, Claude Code).

GLM-5.2 and Gemini 3.1 Pro were able to beat some levels in Tier 4.

Overall, this seems really hard!

“Everyone will have an exceptionally capable personal agent that understands you, your goals, and everything you care about.”

“Everyone will have incredible tools for creation to express your ideas.”

“Everyone will have powerful tools to create new businesses and the economy will become more entrepreneurial.”

“Everyone will have a personalized tutor and coach with a PhD in every subject and unlimited patience to help you learn anything you want.”

“Everyone will benefit from scientific advances and be able to contribute to scientific progress.”

“Everyone will have free or affordable access to these tools.”

The missing question: The part of this essay I understand the least is Zuckerberg’s co-mingling of AI systems capable of invention with individual empowerment. The essay is full of things that seem to assume these things come as a package, for instance:

“While the number of questions a person can ask in a day is limited, the number of valuable things superintelligence can invent to help achieve your goals is unlimited”.

“The more superintelligence serves as a tool of invention, the more likely that individual capability outpaces automation and the future is better for people.”

“Everyone will soon have invention superpowers.”

“Which outcome we get depends on the balance in progress between automation on one side and individual empowerment and invention on the other.”

Things that inspired this story : The construction of the pyramids; Gaudi’s Sagrada Família; tombs and future tombs. Thanks for reading!