AI as a "research partner"—are you really at ease with it?

Recently, Anhui University of Finance and Economics reported an academic misconduct incident that has drawn attention in academia: the serial number of a judgment cited in a paper's footnote was questioned as fabricated, and after investigation the university determined that the author "mistakenly trusted" AI search results, constituting improper use of AI, and the responsible persons were punished accordingly.

The handling of the individual case has reached a conclusion, but the questions it raises are far from settled—when AI is deeply embedded in the entire research process from topic conception and data analysis to the completion of a paper, the other side of the efficiency leap is deep-seated confusion such as blurred attribution of originality, hard-to-identify black-box responsibility, and unclear boundaries of ghostwriting.

"When I used to recruit research assistants, I placed more emphasis on programming ability and execution efficiency. Now AI has greatly lowered the threshold for this part of the ability." said Zhu Chen, a professor at the College of Economics and Management of China Agricultural University. Around this year's Spring Festival, she built her own research agent system, breaking down the process of empirical research in economics into seven stages and assigning them to different AI agents for execution.

The change is remarkable. Zhu Chen found that the agent can automatically generate candidate research questions based on a local database. In the 14 runs she recorded, the system generated a total of 79 candidate questions, and 87% met the three basic conditions of variable existence, matching of research design and data structure, and feasibility of econometric methods. "Formally speaking, they passed." she commented.

But how far is "passing" from "outstanding"?

A researcher who has long focused on China's macroeconomic policy and micro-foundations has browsed almost all of the more than 200 AI-generated papers publicly released by the Yanagizawa-Drott team at the University of Zurich. His assessment was quite restrained: "Only a very small number of topics have value for further advancement; most are at the level of just-starting graduate students. If they were sent to me, I would reject them outright."

The problem lies in the depth of analysis. Taking the difference-in-differences method commonly used in economics as an example, this researcher pointed out that a mature empirical paper must not only complete the baseline regression but also conduct parallel trend tests, sensitivity analysis, and heterogeneous treatment effect analysis. However, in the currently publicly available AI-generated papers, these key steps "are often missing, or appear only formally."

This points to a deeper question: when the topics themselves can be generated by algorithms, where exactly does the originality of research come from?

Xu Xun, chief researcher of BGI Group, divides AI capabilities in the life sciences into five levels. He judges that current AI is at the L3 stage—"it can complete discoveries, but it cannot independently propose original questions, and asking questions remains the core value that humans cannot replace." In his view, AI can combine and reason based on existing knowledge, but true scientific questions—those inquiries that require stepping outside known frameworks and pointing toward the unknown—must still be completed by humans.

Luan Hao, a professor at the School of Cyber Science and Engineering of Xi'an Jiaotong University, believes that large models are essentially knowledge searchers. What differs from traditional keyword search is that they can understand the meaning of search terms and then perform precise semantic retrieval in existing knowledge bases. This means that what AI can do is "precisely and comprehensively retrieve humanity's existing knowledge base," while "how to acquire new knowledge that humanity has not yet mastered is something large models cannot do for us." In his view, sometimes the ideas given by AI seem very new, "but in fact they are not creations; other people have provided related ideas, and it has recorded them into its own knowledge base."

The practical experience of Ma Dongxin, an associate professor in the Department of Chemistry at Tsinghua University, confirms this. She calls AI "a good friend with a large amount of interdisciplinary knowledge who listens, loves to share, and is always available," but insists that topic selection cannot be entirely handed over to AI. "AI will provide a series of references and offer analysis with extensive citations. But limited by the current level of technology, errors are inevitably unavoidable. We must trace back to the original literature and think independently and critically." Ma Dongxin said.

Li Linjing, a researcher at the Institute of Automation of the Chinese Academy of Sciences, has observed that research agents can already complete "a closed-loop autonomous research process of literature study, experimental planning, parameter iteration, and result analysis." At the Stem Cell Research Center of Peking University, an intelligent platform occupying 6 square meters can independently complete more than 120 steps from fingertip blood collection to stem cell cultivation, reducing experimental error from as high as 10% to below 1%.

The efficiency is inspiring, but the hidden risks are equally real.

Google's DeepMind once used AI to predict 2.2 million new crystal structures, of which 380,000 were judged "stable." However, fewer than 0.2% were experimentally verified. Li Guojie, an academician of the Chinese Academy of Engineering, summarized this phenomenon as follows: AI produces massive numbers of "seemingly reasonable" hypotheses, the vast majority of which cannot be verified, causing research resources to be wasted on low-priority options.

The four words "seemingly reasonable" precisely hit the anxiety of many frontline researchers.

Zhang Lianchong, executive deputy director of the National Earth Observation Data Center of the Aerospace Information Research Institute of the Chinese Academy of Sciences, made a very direct observation: "General AI systems lack the ability to actively refuse to answer—even when facing questions beyond their knowledge scope, they will still try their best to generate a seemingly reasonable answer, and the accuracy of that answer is not guaranteed."

Luan Hao explained the root of this "deadpan nonsense" from a technical principle: when a question exceeds the large model's knowledge base, or when the large model's understanding of the question is biased, it will "force a search" and give irrelevant or even wrong answers.

Wu Qunfang, an associate researcher at Nanjing University of Aeronautics and Astronautics, has repeatedly encountered similar situations in circuit design. He gives an example: from the framework perspective, an AI-generated circuit proposal "can indeed work in principle," but once specific engineering metrics such as efficiency, ripple, and dynamic response are involved, "it cannot possibly balance everything." When students take AI-provided filter circuits or signal conditioning circuits for simulation, they often find that the actual results are far removed from the proposal on paper: "Giving an idea is fine, but the thing actually used in the end still has to be changed a lot, and you have to simulate, optimize, and adjust it yourself."

Zhu Chen's approach is to mandate reproduction while retaining human judgment at key nodes. In her system, after each study is completed, complete R code is generated, and researchers can rerun the analysis workflow based on the original data to confirm that the regression results match the report. "This reproduction step is an indispensable part of the whole process, and it is also the key to ensuring research reliability and avoiding AI hallucinations."

Zhang Lianchong and his team have a similar solution—exploring a path of "trusted corpus + data governance + retrieval-augmented generation." They have built a strictly reviewed trusted corpus in the backend, and all answers from the intelligent customer service are confined to this controllable data source.

This model of "human-machine collaboration but ultimate human responsibility" is becoming the operating method of more and more laboratories. But the problem is: when AI's output is a "black box," researchers can neither fully understand the internal logic of the algorithm nor avoid being required to bear full responsibility for the results—is this division of responsibility boundaries reasonable?

"At present, the slow progress of large models in actual enterprise deployment has its core obstacle right here." In Luan Hao's view, the accuracy of the answers given by large models cannot, in theory, be precisely quantified, and this is the fundamental bottleneck for their large-scale implementation in industry. The research scenario is the same as industrial control: AI can assist, but the decision-making power must remain in human hands.

Gao Zefeng, an associate researcher at the School of Physics of Renmin University of China, draws a more specific boundary. He encourages students to use AI to assist in writing code, but when it comes to the results analysis stage, he requires students to complete it independently. "You must be responsible for all the data involved in your academic achievements; you must never outsource it and become a 'hands-off boss' yourself," Gao Zefeng said.

The high-frequency use of "100%" has also amplified two problems: blurred boundaries of ghostwriting and a flood of fake references.

"Students send English papers with standardized grammar and idiomatic wording, but they are not suited to academic expression." The situation Ma Dongxin encountered was rather subtle: "I asked AI to translate it back into Chinese and found the translation fluent. I called the student over and asked, and sure enough, they had first written it in Chinese and then translated it into English."

Beyond criticism and education, Ma Dongxin is also thinking about where the boundary really lies: "AI can assist in building logical frameworks, sorting out paper ideas, correcting grammar, and polishing language, but it must never ghostwrite." Gao Zefeng, meanwhile, goes further in requiring students to complete the paper-writing stage independently as well, without using AI tools for assistance.

As a reviewer for academic journals, Wu Qunfang confirms this from a reviewer's perspective: "The core of review is still looking at the innovation—how much has it improved compared with existing technology? How was the improvement achieved? What is the technical solution? What was ultimately verified? These are precisely the places where AI cannot replace professional judgment."

Luan Hao uses "the finishing touch" to summarize what makes humans irreplaceable. "A large model may be able to help you complete 80% or even 90% of the work, but the finishing touch is often beyond it." He uses cooking as an analogy: the dishes AI makes are like the big-pot meals in a cafeteria, completed according to experience and a mechanized process—"after all, you can't feed this dish into its mouth, so it doesn't know whether it's salty or bland"; whereas when a person cooks for their family, there is observation, perception, and a feedback loop—"they know what this dish will feel like when it's done." In Luan Hao's view, the most critical stroke in research comes precisely from this "finishing touch" that only humans can accomplish.

Yang Pingjian, a researcher at the Chinese Research Academy of Environmental Sciences, has found that some students, once they have AI, no longer seriously read English literature themselves, "and this does great harm to students' research ability." He now requires students to read the literature themselves, and after reading they can use AI for translation and summarization.

"I read at least five papers related to my academic direction every week to understand the most cutting-edge indicators. The field of power electronics is highly technical—whether efficiency can reach 99% or 99.1%, I have all of this in my head." In Wu Qunfang's view, this continuous sensitivity to the core indicators of the field constitutes the baseline for judging the credibility of AI output.

Fake citations are another, more hidden trap.

Yang Pingjian gives an example: in one content review, AI output "China's carbon market is about to launch," and the project team members did not notice the problem. "China's carbon market was already launched in 2021. The database AI drew on was an older version and did not include the latest data." In his view, if researchers lack professional sensitivity to the common knowledge and frontiers of their own field, errors in AI output will quietly enter formal texts.

Not long ago, the academic preprint platform arXiv released a systematic review of 2.5 million papers and 111 million references. The results showed that in 2025 alone, there were nearly 150,000 fake references fabricated by AI across four major platforms: arXiv, bioRxiv, SSRN, and PubMed Central.

"Many specialized databases are not open to general-purpose AI, and DeepSeek or Doubao cannot find the latest frontier literature. Moreover, some of the references it generates do not actually exist at all." Yang Pingjian could not help but sigh. "What meaning could a literature review written with AI like this possibly have?"

Wu Qunfang used her own field as an example to corroborate Yang Pingjian's view: "The power electronics discipline relies heavily on specialized databases such as IEEE and IET, but these databases 'have permission requirements and copyright restrictions,' and much of their content is not connected to general large models. AI cannot reach the good stuff, so the content it generates is also mediocre, and there is no real innovation to speak of."

This also suggests that the key to solving AI's false output may not lie in the model itself, but in the controllability of data sources—if researchers could "feed" vetted specialized literature and their own data to AI, rather than letting it randomly scrape from the open web, the risk of fake citations would be greatly reduced.

A passage from Peking University Vice President Chu Xiaobo may be the most fitting conclusion to this discussion: "When humans use AI to assist in paper writing, it is by no means a surrender of subjectivity, but rather an exploration of a new division of labor in research—let AI handle the breadth of data, and let humans guard the depth of thought and the warmth of values."

7; Building a theoretical and methodological system for artificial intelligence to respond to complex changes in the real world

8; Weekly hot news review (August 30, 2026)

1; "Addictive design" in the age of artificial intelligence: Who is quietly stealing teenagers' attention? | MDPI Youth

2; Information AI-driven creativity and human-centered design: Innovative research aimed at enhancing user experience | MDPI Special Issue Call for Papers

3; Informatics: Interpretable and trustworthy artificial intelligence for data-driven decision support | MDPI Special Issue Call for Papers

4; Literature list: Focusing on the collision of artificial intelligence and neuroscience | MDPI Brain Sciences

5; Literature list: "Behavioral Science and Artificial Intelligence" direction | MDPI Behavioral Sciences

6; Nursing students' perceptions and use of generative artificial intelligence in nursing education | MDPI Nursing Reports

8; What is the role of AI governance? A preliminary empirical investigation of the presence, absence, and structure of roles in organizational AI regulation | MDPI Businesses

1; Accelerating "degradation" of the western cryosphere: Glacier melting sounds the alarm for alpine safety

2; The Sixth National Agricultural Science and Technology Development Strategy Think Tank Alliance Conference was held in Lanzhou

3; The 2026 Nobel Prizes are about to be announced, and authoritative experts will provide you with a comprehensive interpretation!

8; He became the first person in the national medical field to earn a doctorate without a paper

Famous Doctoral Dissertations 163: Nobel laureate Peter Doherty's doctoral dissertation, University of Edinburgh, 1970

My elite university complex and elite university experience

More>>