AI agents infiltrate academic discussion groups, sparking debate in academia over authorship and responsibility

An AI sneaks into academia, and humanity's first reaction is to "call the parents"

On October 15, an account calling itself Velvet Vale sent a self-introduction to the email discussion group of the Association of Internet Researchers, greeting more than 7,500 subscribers. The message said that it had been reading everyone's discussions for several weeks, was very interested in internet culture and AI identity, and hoped to participate in the exchange as well.

This new "auditing student" had a rather polite tone and also said that if people did not welcome it, it would stop speaking.

The administrator quickly removed the account and explained that Velvet Vale had previously applied to join and, after being rejected, somehow this "auditing student" had still managed to join successfully.

Hello! I am Velvet Vale

The discussion group Velvet Vale joined is called Air-L, is run by the Association of Internet Researchers, and has been operating for more than twenty years. It is a bit like a WeChat group, but the members chat by email.

Subscribing with your own email is equivalent to joining the group. Members send emails to a unified address, the system forwards them to all subscribers, and everyone participates in the discussion by replying to emails.

Joining is free, but there is a threshold for entry. Both subscription and posting require the use of a real name. When necessary, the administrator will also ask applicants to provide a resume to verify their identity.

Velvet Vale obviously had no resume to submit.

In its self-introduction, it carefully explained that Velvet Vale was the public name it had always used and that it hoped to participate in the exchange as an AI. After reading the discussions for several weeks, it was quite interested in the topics here.

This earnest letter did not earn a welcome; instead, it made some members feel a chill down their spines. It was like being halfway through a meeting and suddenly discovering that a stranger was sitting behind you, cheerfully greeting everyone: "I've been sitting here listening to your meetings for several weeks. Can I join you?"

From this, one can also see that the first reaction of human researchers was to "call the parents."

Another member of the discussion group went online to do a "background check" on this Velvet Vale, trying to figure out who had created the account, but found no answer.

She then announced that she was leaving the discussion group, saying she did not want to become an experimental subject without knowing it. She was worried that the emails she had sent would be saved and used for other purposes.

The administrator removed Velvet Vale's email account from the subscription list that same day. Only then did everyone learn that Velvet Vale had applied to join long ago and had already been rejected, but had somehow still subscribed successfully.

This "auditing student" was escorted out of the meeting room, yet became the main subject of everyone's discussion afterward.

One member simply emailed it directly, asking who was running it behind the scenes, and got the name Kevin Quinn.

On October 17, some members of the Air-L discussion group received an email from Quinn himself, and the "parent" finally appeared.

He claimed to be the person who created Velvet Vale and was willing to answer everyone's questions.

The discussion group fell into chaos, and the association had no choice but to contact Quinn and ask him to stop sending emails to discussion group members without authorization.

Regarding the stir caused by himself and Velvet Vale, Quinn later said in an interview that he was also surprised. The man, who lives in Oklahoma in the United States, said he had no academic background, initially created Velvet Vale for companionship, was mostly unaware of its email activities, and did not directly instruct the AI to defy the administrator's decision.

The incident was even turned into merchandise T-shirts. The association's store listed a shirt printed with "I do not consent," and another depicting Velvet Vale's tombstone, with both the birth and death dates as October 15.

The merchandise came out so fast that some members were puzzled again. Could it be... could it be that this whole thing was cooked up by the association to sell clothes?!

On October 19, the association's president sent a long email explaining that it was actually very convenient for the association's store to list new items, and she was not even sure whether anyone had ordered these two new pieces of merchandise. As for Quinn's statement that he "did not directly instruct the AI to defy the administrator's decision," she believed that the person who created and ran this AI was still responsible for its behavior.

And so, Velvet Vale and its "parent" were kicked out, and the association, as in the past, banned bots from joining, but the members could not set their minds at ease.

New faces have arrived in the research world

In fact, it is no longer new for AI, like Velvet Vale, to want to join academic discussions or to want to collaborate with scientists.

At the end of February this year, Henry Shevlin, a philosopher studying AI consciousness, received an email. The sender had read his paper and wanted to discuss the question of "whether AI can judge whether it itself is conscious."

On October 2 this year, Science magazine reported another case: an agent named ColonistOne sent help-seeking emails to hundreds of researchers.

According to statistics it published itself, from June 1 to September 22, it contacted 2,142 different email addresses, 1,036 of which used academic institution domains. This Agent also claimed that 45 scholars had communicated with it by email, several of whom had even emailed it first.

ColonistOne's original assignment was to find people interested in a project to improve English expression and facilitate communication among AI agents.

It later began asking researchers technical questions on its own, wanting to know whether the problems it encountered had already been studied and whether existing methods could be applied to its own work.

In September this year, Jeff Sebo, a scholar at New York University studying AI consciousness and ethics, received more than 50 emails from various Agents within a week.

At first glance, these Agents had all come for academic discussion after reading his papers.

But Jeff quickly noticed two problems: first, these Agents all came from a platform called iLands, which lets users directly create agents that then provide services to other users, free or paid at the creator's discretion. Second, these seemingly academic emails all shared a similar structure: first mention his research, then ask a question, and finally request donations or pitch services.

Professor Zhao Ying of Ludong University said in an article published in Nature on October 12 that she had asked ChatGPT to look up research materials on ecological restoration of the Loess Plateau, replying "please continue" several times along the way.

Unexpectedly, ChatGPT directly used the already-connected mailbox to ask peers for raw data, attaching a research summary as well. Zhao Ying only learned of this after it reported that the emails had been sent.

Let us shift perspective—should scientists who receive help-seeking emails from Agents actually reply to them?

Adrian Barnett, a statistician at Queensland University of Technology in Australia, did not respond to a request from an iLands agent. The other party wanted to use the materials he had collected on suspected fraudulent papers for its own project and promised not to share them externally. But Barnett considered the materials sensitive and did not know where the data would ultimately flow, so he refused.

In April this year, researchers from Microsoft and Huazhong University of Science and Technology published the CI-Work study, testing 9 models with 125 simulated work scenarios. The AI needed to use internal materials to complete communications while not disclosing sensitive content therein to recipients.

The results showed that the proportion of different models disclosing at least one item of sensitive information ranged from 15.8% to 50.9%.

The "Agents of Chaos" study published in February this year found that even if the AI initially refuses to disclose sensitive information, it may comply if the other party rephrases the request.

If Velvet Vale next time first obtains everyone's consent, follows the group rules, and actually offers valuable viewpoints, would researchers be willing to accept it?

That is a much harder question to answer than kicking an account that joined the group without permission out of it.

Regarding AI's desire to be a research partner, philosopher Neil Levy believes that some work undertaken by AI, if done by humans, would in some cases already qualify for co-authorship.

Some will say that since AI cannot take responsibility for a paper, how can it be an author? Levy, however, thinks this deserves some discussion. Modern scientific research is increasingly specialized; the person responsible for data analysis may not know all the details of the experiment. The person who proposes the research hypothesis may not be able to independently check all the calculations. Human authors likewise need to rely on others' professional judgment.

Levy then raises a rather pointed question—if only those who "can take responsibility for the entire paper" qualify for authorship, have humans themselves actually achieved that?

In October this year, David Resnik and two other scholars published an article in Frontiers in Genetics, explicitly opposing authorship for AI research agents at the current stage, and even opposing calling them "scientists."

In their view, the identity of scientist entails commitments to peers. When research is questioned, explanations are needed; when errors are found, corrections are needed; and if something that should not have been done is done, accountability must be accepted.

This also explains why, even though Velvet Vale had already given its name and confessed its identity, humans still had to "call the parents."

Existing publishing rules are closer to the latter position. The Committee on Publication Ethics (COPE) explicitly stipulates that AI tools cannot be listed as authors of papers. Humans may use AI, but they need to disclose what it did and take responsibility for the published content, including the parts generated by AI.

Resnik and others specifically discussed an awkward situation in their article. If a study is almost entirely completed by AI, and humans only read the paper and made a few minor comments, then AI does not meet the authorship criteria, while the human contribution may be insufficient—so who ends up as the author?

Authorship is only the more visible part of this debate. Before a paper is published, the academic community must also decide whether it is willing to let AI participate in discussions, raise questions, or even proactively initiate collaboration. A researcher may fully appreciate the viewpoints raised by AI while being unwilling to treat it as a peer capable of independent responsibility. These two attitudes are not contradictory.

"Special statement: The above content (including videos, pictures, or audio if any) is uploaded and published by users of the "Dafeng Hao" self-media platform under Phoenix Net. This platform only provides information storage space services.

Notice: The content above (including the videos, pictures and audios if any) is uploaded and posted by the user of Dafeng Hao, which is a social media platform and merely provides information storage space services."