Oh great, readers preferred AI-written short stories over one by my favorite author in a blind test | We’re collectively awful at identifying AI writing.
https://www.pcgamer.com/software/ai/oh-great-readers-preferred-ai-written-short-stories-over-one-by-my-favorite-author-in-a-blind-test/
25 Comments
“I hate that AI can do this,” is my opinion for most every value of “this” that I’ve seen in the past couple years. It’s also what fantasy author Mark Lawrence said after pitting AI writing against flash fiction written by award-winning authors and finding that not only can readers not reliably tell the difference in a blind test, they also preferred the AI-written stories.
Earlier this month, Lawrence composed a quiz on his blog (which you can still take) with eight very short fantasy stories of about 350 words, asking readers to rate each story’s quality and whether they believe it was composed by a human or AI.
Four stories were written by AI and four by other well-known and award-winning authors. According to [Lawrence’s results post](https://mark—lawrence.blogspot.com/2025/08/the-ai-vs-authors-results-part-2.html), the aggregate decision of the 964 voters correctly judged the origin of three stories, got three others wrong, and couldn’t meaningfully decide on the remaining two, which is not an inspiring success rate. It’s armchair statistics, but that’s not a *small* sample size.”
Why are people so weeeh weh about a.i writing when it obviously has tremendous benefits, like speed of writing through prompt, finding the right phrases and words you can’t think of yourself (especially as a non native-English speaker) etc. Plus you don’t have to literally copy it 1 on 1 but use it more as a basis.
I think that is honestly pretty awesome. It goes to show how much “AI” has progressed in a short time. I’m not too interested in this ridiculous doom and gloom that has taken over Reddit.
I’m very interested to see what will happen over the next 10 years. Maybe we’ll all be able to express ourselves much more creatively, thanks to “AI”.
It does however have downsides too, lots of AI generated turds out there. Hopefully we can also get some proper detection that can remove it from places such as Reddit. Generated content shouldn’t be a thing on a forum. Generated content should also be clearly marked, so everyone can see that it is.
It’s not that readers are awful at identifying AI, but rather that AI has improved so much that it can hardly be discerned from human writing except by the sharpest and keenest minds
[deleted]
A lot of people prefer Taylor Swift to actual music … People like common demonstrator slop. What’s new?
This seems explicitly like the kind of edge case test AI could succeed in though. Short form with no longer context needed, no depth of character development or storytelling required, and isolated in a list which always makes comparative analysis harder.
But also who was polled? Folks who have been using AI the past few years likely can recognize it better than those who haven’t. That’s most certainly true for images and video, seems reasonable to assume there is an “AI illiteracy” issue with large amounts of the public.
And finally, oh well? As language models get better and better it’d be kinda silly to assume they wouldn’t be able to write enjoyable prose. A primary reason for their existence is they “contain” massive, inhuman amounts of accumulated human knowledge and language trends in their “minds”, this should be considered a natural consequence.
Edit: just took the test, got 6/8 correctly identified. Mistakes were made on the last few stories, which for me confirms the “comparative analysis of a list” issue.
This is stupid.The AI stories were at a minimum created by crafting a good and relevant prompt. He most likely also “tuned” the output afterwards.
Basically AI at this time works because you have human directed input and human chosen output.
All of these stories were pretty bad though. I mean, none of them had anything interesting to say. Some had better sentence construction and so on, but they were all lousy.
I wonder what AI they’re using because I sometimes write short stories and use chat gpt to write another one to kind of bring up things that I don’t think of and it always produces results which are weird as it obviously doesn’t have a theory of mind so it’ll do something and then think other characters immediately know about it without being told.
All this proves is that readers prefer a good plot over the quality of the prose.
Or genre fiction of 350 words or less isn’t really the format of great literature to begin with. It’s like saying that they did a test where they pitted 5-minute CGI action scenes created by human beings against those made by AI and the results were mixed on which scenes viewers preferred. Yeah, no kidding.
AI produces bland, average, innocuous stuff. Not surprising if some people prefer that. Taste is not distributed equally.
Short texts are best case scenario for AI. Generating fictionally text is also best case for AI. It’s different story when AI is writing non-fictional stuffs….
AI, in the end will be better not just any human, but all humans combined. We aren’t there. Not even close there. Yet. It may happens in distant future. But it could also happen in decade.
The problem with AI is similar to the problem with Trump.
You can say “oh well if idiots fall for that shit too bad for them”
But they only need like 30% of people to not give a shit if it’s ai or not and it can come to dominate society.
That’s what 1984 is actually about – the fact the ruling elites only need to trick the proles. Then the intelligent people will be at risk and unable to protect themselves.
I don’t want to read any text or see any ai images ever again.
When creating art, there are always two aspects to it. The technical craftsmanship, and the creativity. AI is very good at the former, but not the latter. The way this test was set up, it mainly evaluates the craftsmanship element, not the creativity one. This is no different to photography making painters somewhat obsolete. No one will argue that this was a net positive on society. With AI it will be the same.
That’s because AI is optimise to produce what the majority of people would like in the immediate term, irrespective of any objective quality.
A human author may try to promote a moral, or challange their readers, or just write what they like. An AI just writes what creates the most engagement with it.
A comparison. Before we made precise machines that could sew a shirt, they were all sewn by hand. Part of being handmade are all the little errors and imperfections. Now all the shirts are made by a machine. Perfectly every time. And people do prefer them. Sure a minority pay extra for “hand made” and espouse how it’s better, but most of us don’t.
When self driving tech is a lot better than it is today, every long range fleet driver will complain that they hate that machines can do it now. Yet they will drive better without distractions that humans have. Loads will be delivered faster too.
People just thought writing, filmmaking, and poetry were safe, but they aren’t.
The real issue isn’t that readers can’t tell the difference but that they have no reason to care. I read for the story and not to validate the author. Is the author a white make, a woman of color, an LGBTQ, or an AI isn’t the question. Is the story compelling? Does the reader engage with the characters. Does the reader see a reflection of themselves in some elements of the story? It sounds like AI is checking the boxes for readers. That’s a red flag for traditional authors and the system that supports them.
Might as well get used to this; as the models get better and better it will happen more and more.
or maybe AI writing is better than your preferred autor
Title has me confused.
Were they bad at figuring out which was AI, or did they just simply prefer the AI stories more than the real one? Preferring AI doesn’t mean they couldn’t detect it.
The popularity often doesn’t equal that a story is good, only that it’s entertaining. If you are looking for entertainment, it’s no big surprise that a machine can imitate proven formulae, but if you’re looking for art and to broaden your horizon, opinions on a story will be divided, because it challenges readers to some degree.
I’ll do the test myself later to see how much the stories can move me (and how well I spot ai content).
On morals I don’t read AI writing.
I won’t be bothered to read something someone wasn’t bothered to write. I want a worse writing made by humans and I’m not joking.