Hallucinated citations are polluting the scientific literature. What can be done? – Tens of thousands of publications from 2025 might include invalid references generated by AI, a Nature analysis suggests.
https://www.nature.com/articles/d41586-026-00969-z
36 Comments
Citations should be checked by reviewers and submitters should be banned for making things up. If enough submissions from a given institution are rejected for making up false information, the institution should be blacklisted.
It shouldn’t be hard to automate a simple check for whether cited works exist or not
What I don’t understand is that often I’ll use GPT to find quotes and sources for a reddit comment. GPT will say exactly what I want and provide sources, but when I actually look at the sources either they don’t exist or don’t actually say what I want.
So it seems like I spend more time and effort checking GPT outputs for a reddit comment than these scientists do for a proper studies.
It makes me question how rigorously citations were verified pre-AI.
> What can be done?
Just peer review the damn peer-reviewed articles?
They won’t just feed articles to AI and expect it to check the citations, right?
… right?
There’s no excuse for entirely fake citations. This is why DOIs and citation management software exists.
Now, citing real articles for fake content is an entirely different story and is much harder to detect or police. Always has been.
Just make the authors who cite non-existent works write the works they cited.
the authors should be responsible for it. if multiple citations are clearly ai generated and fake, the paper should be rejected, and the authors should be banned.
Hard(er) consequences for authors of articles with fake citations.
As long as a significant portion gets away with it until much later, with only mild consequences, the practice will continue.
Especially in an academic setting where number of publications into what kind of journals defines one’s standing to a large degree, which in turn strongly influences availability of funds.
I also hope freely and easily applied technology will continue to be developed to automate citation checking.
As long as GenAI can spit out the false nonsense faster than it can be checked, it’ll remain to be a problem…
Retract the offending articles? Never gna happen though
If it’s written by AI it’s not scientific literature.
Generated by ai is a weird new take on an old problem. Academia has always been full of fraud. Peer review has become an self-serving exchange with both getting credit they don’t deserve.
Maybe check the references before publishing stuff lol
The “AI efficiency increase” is only counted by producing results faster than before. What is totally overlooked that you need much more time to verify and correct the results manually.
(unless your confidence is as big as you ignorance and expect everything produced by AI to be correct)
I am totally surprised scientists did not proofread their papers for fear of being caught using AI or for errors. I don’t have a problem with the using AI but IMHO they should have reviewed the work before submission.
Isn’t this what peer review is for?
What the fuck are we doing here?
The answer to this problem is obvious — any one who submits a publication must declare the use of AI and, if found to include a hallucinated cite or false fact the published authors get a 10 year ban from publishing. In a publish or perish environment, it would basically be the death penalty for second and third authors and this strong motive not to fuck around.
Stop using AI?
I mean, at uni, if my APA 7 citations are wrong, I get dinged pretty hard marks wise. Why is it any different out in the academic world?
I sometimes use AI to find source docs for citations i cannot find or remember. Often more effort getting ai to tell the truth than is worth it
If you publish something under your name with fake citations, I don’t care what software you used to write your paper. You should probably be barred from publishing in reputable journals again.
Identify the first use of fake citations and bollock the authors.
Can we call these “halucitations”? Please? Thank you.
Name and shame the “scholars” who submitted this slop. There needs to be reputational and career affecting consequences.
Everybody talking like, “what peer review doing?” which is valid. But the real culprits are the grifters thinking they could get away with having ChatGippity write their articles.
I doubt this will be a popular opinion, but I think this is more harmful to the reputation of science than it is to the quality of science itself.
The much of the front end of a paper is spent (1) setting the context for the research question and/or (2) arguing why prior research was inadequate. If there are a few fake citations here, it doesn’t matter that much. The table for the research question is already set. The research question is posed, the methods and data are defined.
Also, peer review is done by domain experts -adolescent psychology focused on ADHD or whatever – who know what’s most important in their field. They all know the big papers that need to be discussed. Letting a fake 3rd citation on a claim about X slip by doesn’t impact the other two. the most important ones are there already.
I think that scientists need to write their own lit reviews and not like AI stuff citations, but I don’t think this makes for bad science by itself.
dead internet theory but for academic papers
Just check your citations before submitting anything? Easy-peasy
Last week I read women suffer from higher rates of visual impairment than men… and had to double check every single statistic they had drawn their information from, as I have heard the exact opposite my whole life . They were willing to draw from multiple visual disease and impairments, but forgot colorblindness was a thing.. and not a single one of these studies pointed out the exclusion of colorblindness (which is classified as a visual impairment). None of them even bothered to emphasize a statement like “under these conditions” when looking across age ranges or environmental factors.
Anyways… point I am making is, if you’re worried about scientific studies being bogged down by bull-shit… you lack the critical thinking skills necessary to become a scientist anyways.
Start fining people who submit papers that have Ai generated citations.
You’ll never find a place that hates technology more than r/technology.
If only we had a technology capable of reading, digesting and validating the connections between citations and publications.
Why are the sources of truth using AI? That’s such a bad idea. That’s about as far upstream as you can go.
Once the hallucinogenic citations are cited hallucinogenically for the next round of publications they will all cancel out.
Mandating all references to mention their doi (https://fr.wikipedia.org/wiki/Digital_Object_Identifier ) is an easy solution: it should make it easier to verify that the citation is correct.
I have chronic back pain and I asked Chat for some info. It cited a paper that sounded relevant; I went out to PubMed to pull the paper and there was nothing by that title/subject in the journal. So, I went back to Chat and it said something like my mistake I’ll be more careful.
Shut it all down.
This problem existed before LLMs and extends beyond non-existent citations into citing work that does exist but then drawing the wrong conclusions (claiming the sky is green and then citing a paper that shows the sky is blue.) The thing is, this can all be checked systematically and LLMs can even help solve the non-mechanical part that has been too onerous for peer reviewers in the past. I imagine all reputable journals will need to put these sorts of systems in place, and according to the article this is already happening.
It’s almost like someone should be checking this stuff and I don’t know looking up the references? People are fucking looking for any excuse to not do anything anymore. Smh