AI citation bias: who did the machine leave out?
I have a confession that sounds almost silly in 2026: I don't let AI anywhere near my reference lists.
Friends think I'm being precious. Everyone I know in research uses ChatGPT or Claude somewhere in their reading workflow now. Ask for ten papers on any topic and you get a tidy, plausible list in seconds. It feels like getting an afternoon of your life back.
Then I spent a few weeks reading the new research on AI citation bias. My full write-up is on my Substack, but here's the short version, because it matters well beyond academia.
What the studies found: gender bias in AI-recommended citations
Two research teams tested what happens when AI models choose citations. One ran a controlled experiment where identical papers were labelled with male or female author names. GPT-4o picked the male-named versions at significantly higher rates, in every condition tested. And when women's papers were the smallest minority in the pool, the model was most likely to skip them entirely. Men's papers in the minority didn't get the same treatment.
The other study found GPT-4 recommends papers that are already heavily cited, by a huge margin, even after controlling for the obvious explanations. The famous get more famous. Even the references the model made up (they still do that) followed the same pattern.
Why AI bias hits women of colour hardest
None of this bias is new. The machine learned it from decades of human citing, and humans have been under-citing women and scholars of colour for as long as anyone's been counting. What's new is the scale. A biased colleague influences one reading list. A biased model sits inside every reference manager and search tool, shaping millions of lists, then feeding the results back into the next model's training data.
The detail I can't stop thinking about: these systems punish rarity itself. Whoever is scarce in the pool gets overlooked, whatever made them scarce. Women of colour are scarce in almost every pool a model will ever see. The people academia already pushed to the margins are the ones the machine is quietest about.
Why I still build reference lists by hand
It's a small habit, and one I often pay my RAs to do, and I know it doesn't scale. But I can't audit what a model quietly left off my list, and I don't trust the omissions. When we build a reading lists by hand, at least the prejudices in it are mine or my staffs to notice and fix.
The full piece has the numbers, the studies, and what I think should change (disclosure rules, and a reckoning with citation counting itself). Read the full article on my Substack.
And next time an AI hands you a neat list of sources, ask it my new favourite question: who did you leave out?
