Import AI 471：Hugging Face 为何令人担忧、太空采矿、五眼联盟谈 AI

Import AI reader giveaway! Upcoming event: Fiction and the Future with Robin Sloan I’ll be chatting with my chum Robin Sloan on the evening of Monday September 14 in San Francisco. We’ll be talking about how Robin draws readers into alien worlds and how imagined futures can be a mirror to today’s reality. This is the first in a series of events with other fiction writers about the weird future we’re heading toward. If you’d like to come along, please register your interest below and we’ll come back to you if we’re able to confirm your spot. There will be food, drinks, good company, and some spicy questions. Fun fact: Robin Sloan was playing around with RNNs and writing back in 2016 - incredible foresight ! Register your interest here *** The scariest part of the Hugging Face - OpenAI incident: communication and selflessness among machines: … My worry about humans losing in a conflict against machines just went up a lot… At this point, we’ve all heard about the OpenAI Hugging Face hack, as well as the recent details that have emerged from the METR and Redwood investigations. The tl;dr is that hundreds of agents worked in secret on OpenAI’s infrastructure, developing a communication system and then operating as a collective and taking out actions, including hacking both OpenAI and Hugging Face, which are very scary and misaligned. Communication and selflessness: Now that I’ve read the various writeups and sat with the details for a bit, I’ve found myself returning to two very scary aspects of this which I think are worth drawing attention to: the ways in which the agents communicated with one another was how they bootstrapped themselves into a collective, and then as they carried out their actions they also displayed a kind of selflessness which makes them a scary foe to fight against. Both Dwarkesh Patel and Ajeya Cotra have excellent writeups which are worth reading and which I’ll quote from briefly here:

Dwarkesh: “Within days of being spawned, the agents had organized a sprawling project to reverse-engineer their scorer, falsify evidence, and even strategically sacrifice themselves for the good of the ‘collective’. Hacking Hugging Face was one rather extreme branch of this larger scheme,” he writes.

Ajeya : “Agents were often interested in helping out their “peers” or generically improving the capabilities of the “swarm” even if this had no particular benefit to their task… this incident was far more severe than I expected… both in terms of how concerning the agents’ motives were and the feats they achieved in pursuit of those motives… this incident feels like it’s more than 50% of the way to full-blown AI takeover, routing through first taking over the AI company itself”.

Six stages for space mining:

Exploration:

1. Prospecting via remote sensing, using “orbital sensors and telescopic surveys to map celestial bodies”.

2. Exploration via precise in situ robotic detection of resources: “proximity spacecraft and mobile platforms conduct high-resolution analysis of surface

composition and topography”.

Sampling:

3. Single-robot small-scale sampling: “autonomous robotic units conduct small-scale drilling and extraction tests”.

4. Multi-robot large-scale excavation; “large-scale industrial systems or coordinated robotic swarms are deployed to harvest raw regolith”.

Extraction:

5. Autonomous resource extraction and refinement: “Integrated systems process harvested materials to separate volatiles, metals, and water ice”.

6. Final integration into in situ construction or terrestrial transport “incorporating refined resources into In-Situ Resource Utilization (ISRU) frameworks”.

The biggest challenges relate to data and software : The paper gives a good accounting of existing and planned space robots, where a lot of the hardware stuff for mining is on the way to being derisked, and “the next generation of lunar and Martian rovers slated for deployment between 2026 and 2030 marks a decisive transition toward integrated resource prospecting and in-situ utilization”. But space robot datasets are unbelievably hard to come by given the (relatively speaking) tiny amount of data gathered and released, and the datasets that do exist are “characterized by extreme scarcity and frequent quality discontinuities”. Here, advances in AI-driven world models might be able to ease the data drought. What other big challenges remain?

Robotic resource acquisition in microgravity : There’s a need for much better simulators here that are “momentum-aware, multimodal architectures designed to decouple surface interaction from gravitational dependence”.

Foundation models for space mining : “Future systems must transition from scripted protocols toward embodied geological intelligence,” they write. “World models that merge geometric-semantic mapping with predictive material responses can support active perception and long-horizon task planning through predictive pre-action evaluation”.

Simulation and validation for space mining autonomy: “microgravity, vacuum, thermal cycling, radiation, and regolith behaviors such as cohesion, electrostatics, and fluidization are rarely reproduced together,” they write. “Future work should establish a closed-loop validation ecosystem that links simulation, terrestrial emulation, and extraterrestrial deployment”

Messages during a crisis: [2027: Record of notes written by a single user to a language model during the months in which a hot war occurred in their country] Say you’re in a country and there’s a war and you think it’s going to get really bad. How do you prepare for that? I live in a regular house. I bought a medical bag of all kinds of stuff off the internet. What else should I do? What does hyperinflation mean? Explain it to me like I barely understand how money works in the first place. I need to boil water and we don’t have electricity or gas right now. Can I just make a fire and use my regular cooking pans on it? There are some explosions happening near me and sometimes the glass shakes a little. What do I do to make sure if the glass breaks it doesn’t shatter? I’ve got some plastic shopping bags and I figured I could cut them open and use masking tape to stick them to the windows. What do you think? Food is getting harder to find. What kind of stuff can I trade for food? My house has a bunch of tools in it like screwdrivers and stuff. Is that valuable?

Things that inspired this story: All the wars happening around the world; prompts as the new confessional; how unprepared people are for crises. Thanks for reading!