As a writer online and a researcher offline, my work pretty much always involves looking up existing research or literature concerning a certain essay topic or research question. On here, we have discussed in some length the existing controversies and fraudulent practices in research and academia, from generated images and manuscripts, to the questionable practices by peer reviewers and publishers which allow these manuscripts of dubious quality to be published. Today, we will focus on another element in a manuscript that may prove fraudulent.

Firstly, I would want you to look up pretty much any manuscript, and go over the content. Do you see number notations like [1], or mentions of previous research such as (Doe et al., 20XX)? These are called citations, which references a certain point, statement, or part thereof to existing research. It is a common practice in academia, research, and even assignments where one would need to do their own research and reading to write up a certain topic.

Plagiarism is not a novel thing when it comes to these domains. Lifting, low-effort paraphrasing, and the like are the traditional forms of plagiarism seen in these forms of content. However, over the past decade, we have seen an increase in the scale of plagiarised works, catalysed by the advent of large language models and generative artificial intelligence (let us just use the abbreviation AI here), which are trained on data that is questionably or dubiously obtained from various legitimate sources (and some illegitimate ones like paper mills), including journals and publishers. Large language models would pull content from these sources when prompted with a certain question, and would likely plagiarise from these sources by lifting and filling them with style words. As a lazy attempt to mask these cases of plagiarism, some would prompt the large language model to ‘cite sources’. This is where the problem of focus enters the picture.

According to an experimental study conducted by Linardon et al. (2025), it was found that a fifth of all the generated citations were fabricated with almost half of the remaining valid citations containing errors. What I personally find fascinating is the structure of these fabrications, as shown in the supplementary material such as the one below:

From a superficial look, these citations do appear legitimate, with proper formatting following some citation conventions. The authors may have been linked to a related research field as that mentioned in the prompt, and would be included in the citation. These authors may also have worked with the first author before, perhaps as part of the same research group or consortium in past real research. In the titles, the keywords may be related to the authors’ field of research and the prompt. Heck, even the journal, issue, volume, and page numbers may be plausible as well. However, the digital object identifier, or DOI, does not link to a valid publication. If the submitted bibliography does not include this DOI as part of the format, it would be pretty understandable how these hallucinated citations could be mixed with genuine ones. The combination of multiple plausible elements in the citation would produce something that looks real, but more rather, uncanny.

The most intuitive direction to point the finger at would be the peer review process. After all, this is the external quality control part of the research to publication pipeline, where peer reviewers would vet and suggest necessary edits to the manuscript to create a publication-ready one. However, the current peer review situation is pretty tenuous. Workload for peer reviewers are pretty high, and they may be required to vet and review numerous manuscripts encompassing many fields of research in a rather limited time. Delayed reviews may also incur some penalties as well, meaning that it would be in the reviewer’s best interest to simply expedite the process to get through as much as they can. This pressure would lead to the lowering of rigor and quality of the published manuscript. With generated manuscripts and hallucinated citations, it would be inevitable that these would sneak into journal publications like a Trojan horse.

I think what shocks me the most is the scale of the issue. According to this Nature article, amongst 4000 publications chosen as a random sample from publications from the 5 leading academic publishers in 2025, 100 of the most suspicious articles were manually checked for hallucinated citations. It turned out that 65 of these articles contained at least one hallucinated citation, with a further 13 having a rather unclear status. These invalid citations could have been made by human error, or could link to non-English articles, which may have differing transcription methods. Using a very back-of-the-napkin extrapolation, the article suggested that as many as 110,000 of the 7 million academic publications in 2025 alone could contain at least one hallucinated or invalid citation, with most of these being attributed to AI. A preprint study detailing the analysis of around 3,000 conference papers found that 10% of them having at least one hallucinated citation, which the researchers termed HalluCitations, a portmanteau of ‘hallucination’ and ‘citations’.

Now that fake citations are a rapidly growing problem in academia, how do we identify these fraudulent cases?

When researching a topic, two of the techniques used to look for more sources involve finding papers or literature that either build up or lead up to the citation in a paper, or seeing which papers have cited a certain publication. The former method is called backward chaining, where the paper cites sources which have influenced the author’s writing of the paper in question, and these sources are found as citations at a certain statement, and as references in the bibliography. The latter method is called forward chaining, where you see publications that have cited the starting resource or publication (the ‘cited by’ part in the online version of the article), and could be relevant as refutations, replicative studies, and studies that may be somewhat related in findings, methods, or perhaps the discussion.

Usually, fraudulent citations would manifest during the backward chaining process, as the AI would want to come up with something that substantiates a certain statement or argument that it churns out. Thus, during the backwards chaining process, the most intuitive way to verify if the citation is real or fake is to just search up the cited work. Copy the reference into a search engine (not artificial intelligence) of your choice, and see if the journal, authors, article, and other relevant details exist verbatim. Usually, these fake citations have mismatches in at least one of these fields.

So, now that you have verified that the paper exists as cited, the next layer of verification is to go into the content of the study to see if the study cited is actually what the resource says or argues. Misrepresentation, misinterpretation, or fabrication of findings of the cited study are common tells of a hallucinated citation, but they are not exactly watertight. Some human-made citations may be susceptible to misrepresentation of study findings and interpretations. Thus, you might also want to see if the citation is used appropriately. This requires more understanding about the content of the cited study and the context in which it is being used. Given that one might have to conduct these verification for some or even all of the cited papers for just one resource, and that one study might have anywhere from 20 to 50 references for experimental studies, and perhaps more than 100 references in review articles, there would be a considerable burden to the reader or reviewer to go through when reading the article. Therefore, some fraudulent citations would have just made it past the review process. After all, the peer review process is not really perfect, given the track record of approving generated manuscripts for publication. Perhaps we would expect the same when it comes to citations.

Undoubtedly, the worst thing to do is to ask a large language model to verify this for you. Sometimes, these would just perpetuate further hallucinations, saying that a cited work is real when in reality, it is not, perhaps to feed into some form of confirmation bias. Models like these mainly work through statistics and probability, by drawing patterns of how words (converted to tokens) are connected to other words. Thus, its responses are not always in line with reality, and should be treated with utmost scrutiny and scepticism. As such, the advise to ‘do your own research’ has never proved more important than before. Use your brain, put in the effort, do your own research, rather than outsourcing it to an artificial intelligence.

Further reading

Linardon, J., Jarman, H.K., McClure, Z., Anderson, C., Liu, C., Messer, M. (2025) ‘Influence of Topic Familiarity and Prompt Specificity on Citation Fabrication in Mental Health Research Using Large Language Models: Experimental Study’, JMIR Ment Health, 12, e80371. doi: 10.2196/80371.

Sakai, Y., Kamigaito, H., Watanabe, T. (2026) ‘HalluCitation Matters: Revealing the Impact of Hallucinated References with 300 Hallucinated Papers in ACL Conferences’, arXiv. https://arxiv.org/abs/2601.18724.