←News detail

36Kr · AIUpdated Oct 30 16:23

Neuralink Connects Grok to the Brain: ALS Patient Controls AI Agent with Thoughts

AI summary

Neuralink has announced its latest progress: Kenneth, a patient with ALS, silently thinks "Hey Neuralink" and formulates a task in his mind, which then wakes up the Grok agent on his computer to autonomously execute the operation. The team conducted a 72-hour development sprint at the subject's home, producing 8 functional prototypes, including neural voice activity detection (VAD) that requires no manual switch, a system-level brain-computer input method called BCI Keyboard Dictation, and a mobile app called Bloom. Kenneth also used speech generated by the brain-computer interface to connect to the iPhone call audio stream, making a phone call to his wife with his thoughts for the first time in seven years.

Why read this

AI score 71

In a 72-hour development sprint, Neuralink connected its brain-computer interface with the Grok agent, allowing readers to see the interaction chain of thought-driven AI operating a computer.

Topics

Tags

Body · AI translation

The scenes from sci-fi movies The Matrix and Transcendence are being brought into reality bit by bit by Musk.

Today, Neuralink dropped a latest update that is genuinely a bit stunning—

This time, Musk directly hooked SpaceXSI's Grok straight into a human brain!

All the patient has to do is silently say "Hey Neuralink" in their mind, then think about the task they want done, and the Grok Agent on the computer is instantly awakened.

Next, Grok automatically takes over the screen and goes straight onto YouTube.

And the most touching moment of all was a very special phone call.

The chip in the brain of a severely paralyzed, speech-impaired patient directly converted "the words he wanted to say" into a pure audio stream, injected in real time into the underlying microphone call layer of an iPhone.

After seven years, he finally used his mind to call his wife.

Humanity has taken another extremely crazy step toward true "mechanical ascension."

A record of 72 hours of god-tier work, the most hardcore upgrade to "brain-computer interface"

All of this was made possible by a "hard battle" the Neuralink team fought a few days ago.

Several core engineers simply packed their bags and moved into the subject's home, eating and living together, for a full 72-hour extreme lockdown development sprint.

For three full days and nights, they crammed work that normally takes several weeks into just a few hours, grinding out 8 brand-new features in one go.

In the past, our understanding of brain-computer interfaces was still stuck at "using thoughts to move the mouse cursor."

After these 72 hours, the brain has officially become a native input source for AI.

The patient's name is Kenneth. He has been diagnosed with ALS for seven full years and is a Neuralink P21 subject.

Throughout it all he has been especially optimistic, just as he often says, "ALS sucks, but my life doesn't."

Say the word in your mind, and Grok takes over

The most sci-fi feature this time is, of course, Grok.

A brain-computer patient only needs to imagine saying "Hey Neuralink" in their head, plus the task they want completed, to trigger a Grok agent, which then autonomously executes it on the computer.

No hands needed to type on a keyboard, no mouth needed to actually speak—the command is issued from the neural activity end.

Break this chain apart and it becomes easy to understand: the brain-computer interface is responsible for turning expressed intent into usable input, and the Grok agent catches the task and then operates the computer.

The so-called "access" is essentially a complete reconstruction of the dimension of interaction:

Thought-controlled cursor: what the brain outputs are low-level actions like "a little to the left, click once," and completing a task takes dozens or hundreds of steps, with the person having to watch the whole time;

Thought-driven Agent: the brain outputs only a single sentence of "intent," and breaking down the steps and operating the interface are all handed to AI.

The biggest bottleneck of brain-computer interfaces has always been bandwidth: the number of electrodes and the decoding speed both have physical limits.

The value of the Agent lies in the fact that it "amplifies" one sentence into a whole chain of operations—without adding a single electrode, it multiplies what each thought can accomplish.

As Neuralink itself said, this is only an early exploration, but it has solidly proven that:

as long as communication between the brain and the computer becomes smoother, the gap between "thinking of it" and "doing it" can be erased.

Breakdown of the eight core technologies

The 8 technology prototypes born in these 72 hours run from low-level signal capture all the way through system-level interaction, ultimately connecting to the top-layer AI large model.

Neural Voice Activity Detection (VAD): completely throwing away the physical switch

Before this hackathon, if Kenneth wanted to speak through thought, he still faced a rather awkward paradox—you first had to manually press a button to tell the software "I'm about to start thinking," and after speaking you had to press it again to stop recording.

For patients with ALS or high-level paraplegia, pressing this button is itself a huge burden.

To solve this, the engineers directly rewrote a "Neural Voice Activity Detection (Neural VAD)" system from the ground up.

This algorithm is like a neural sentry on duty 24 hours a day, monitoring in real time the stream of brain signals sent back by the implant.

It can extremely keenly distinguish whether the brain at this moment is in a resting state of "zoning out and daydreaming," or has already engaged the motor cortex and is "getting ready to speak."

Once the intent pulse to speak is captured, the app automatically starts decoding within milliseconds; the moment the words end, the system automatically goes dormant.

Zero button presses, zero hand movement throughout—truly achieving pure thought "speak as soon as you open your mouth."

Real-time neural signal visualization: patient and model "train together"

The brain-computer interface field has always had a headache-inducing black-box problem: even the subject themselves doesn't know exactly "how to think" in their head to trigger the strongest, clearest neuron firing.

You see, the higher the signal-to-noise ratio of the input signal, the more outrageous the accuracy of the backend machine learning decoder.

The team turned the signal strength and spatial distribution detected by the implant into real-time visual feedback for Kenneth. He could "pronounce in his mind" in all sorts of different ways and see on the spot which method engaged the most neural activity.

In neuroscience this belongs to closed-loop neurofeedback: not only is the model adapting to the brain, the brain is also adapting to the model in turn.

What was originally invisible and untouchable parameter tuning has become a "level-clearing game" with instant feedback.

Order coffee on your phone, without using your hands at all

Over the past few months, Neuralink has been quietly polishing a dedicated app called Bloom internally, with only one goal: to fit a massive speech brain-computer decoding system into a phone.

Taking advantage of this hackathon, the engineers simply pushed Kenneth out the door, brought Bloom along, and headed straight to noisy public places.

In a real café, for the first time Kenneth did not rely on someone else to relay his words; instead, through Bloom on his phone, he ordered himself a coffee with his thoughts.

And this was only the beginning.

For ALS patients, Apple's built-in software keyboard and Siri voice dictation are almost unusable. So the team directly called low-level iOS interfaces and hand-built a system-level brain-computer input method: BCI Keyboard Dictation.

As soon as the keyboard is brought up, the words Kenneth wants to say in his mind go through neural decoding and flow like water into any app on the iPhone.

Writing memos, replying to messages, browsing social networks—no need to switch back and forth between different modes the whole time; point and go.

Of course, decoding inevitably makes mistakes.

To address this, during the hackathon the engineers iterated through several versions of a correction interface, and ultimately embedded a predictive correction workflow directly into the keyboard: once recognition deviates, several high-probability candidate words immediately pop up above the keyboard, and with a glance of his thoughts Kenneth can complete a "light-speed replacement" in any app.

There is another detail that lowers the threshold for starting everyday communication to almost zero.

The engineers cleverly borrowed the iPhone's Live Activities and Dynamic Island mechanisms to keep Bloom resident on the lock screen.

Kenneth does not need to unlock the phone; when the screen lights up, he can see the current brain-computer connection status; with a flicker of thought, he can start "speaking" directly from the lock screen, truly achieving "instant replies."

Paralyzed for seven years, he finally made a phone call with his thoughts

Among all the features, "making a phone call with thoughts" was not the highest in technical difficulty, yet it became the tearjerker of the whole event.

Technically, the team connected Bloom with iPhone calls, allowing the speech generated by the brain-computer interface to feed directly into the call's microphone audio stream.

But the weight of this matter lies not in the technology, but in the story.

Kenneth and his wife's love began over the phone. When the two first met, their relationship was slowly built through one phone call after another. That telephone line was once the most important thread between them.

Later, illness made it hard for him to speak. That thread also gradually fell silent.

This time, as soon as the engineers finished building the call feature, he picked up the phone and called his wife.

The call connected. He did not open his mouth. The words he wanted to say in his mind were captured by the implant, translated by the decoder, synthesized into speech, and sent directly into the call line.

On the other end of the receiver came his wife's laughter.

The two began to tease and bicker with each other, just like when they were dating over the phone back then.

The engineers just stood by and watched. Neuralink later said this was one of the most unforgettable moments of the entire week.

In engineering, few things are more fulfilling than this. You watch a person use code you wrote that very morning to talk to the person they love.

In the past, the brain's bandwidth was too narrow, and people could only "squeeze toothpaste" to machines one letter or one click at a time.

This will become a major leap in human-computer interaction.

In the future, as electrodes become more numerous, decoding becomes faster, and agents become smarter, "thinking it means doing it" may move from the lab into more people's daily lives.

In The Matrix, Neo plugs into a line and learns kung fu in seconds; in Transcendence, human consciousness connects to the network and can directly mobilize the entire digital world.

These imaginings that once belonged only to the screen are being written into reality bit by bit.

This article comes from the WeChat public account "Xinzhiyuan"; author: ASI Revelation; published by 36Kr with authorization.

Source

36Kr · AI