Arjun Panickssery: Judgment in Autonomous AI R&D Still Lacking

Arjun Panickssery said on LessWrong that across several autonomous AI R&D projects he followed (a Scrabble engine, an Ultimate Tic-Tac-Toe bot, and formal proofs for an AI Philosophy competition), the models' judgment was poor at a basic level: literature reviews missed the most critical papers, experiments were uninformative because the training data was too small, and the philosophy project repeatedly drifted into trivial directions. He considers the exception to be hill-climbing tasks with fast feedback loops—for example, by repeatedly prompting Astra he raised the SOTA for "list all legal Scrabble moves" by 10x, but the two key jumps still came from his own prompts.