The AI detection tool that flagged my own writing as fake

Last month I ran a project report through a popular AI detector to check for false positives, and it came back 92% likely AI-generated - except I wrote every word by hand three years ago for a client in Chicago. I tried feeding it a Shakespeare sonnet and it flagged that as AI too, which tells me these tools are mostly guessing based on writing style. Anyone else finding these detectors useless for actual verification?

2 comments

2 Comments

brooke4751mo ago

yeah the shakespeare thing really gets at it. i mean if these tools can't tell the difference between actual human writing from 400 years ago and something a computer spat out yesterday, what are we even doing here. the whole "92% likely AI-generated" thing is wild too, like what even is the percentage based on. you wrote a project report and it's basically saying your writing style is statistically improbable for a human? that's messed up. so here's my question - did the tool give you any feedback on WHY it flagged it? like specific patterns or word choices it thought were AI-ish? because if they can't even articulate what they're detecting, it's just a black box that makes teachers and managers paranoid over nothing.

the_drew1mo ago

and that's exactly the thing that bugs me most about these detectors, they don't give you any breakdown at all. zero explanation. just a number that looks official but means nothing. i had a friend who got flagged on a paper she wrote completely by herself, and the tool gave her "high confidence" but wouldn't tell her if it was because she used too many transition words or had a certain sentence structure. it's like being pulled over by a cop who won't tell you what you did wrong. the whole percentage thing is probably based on some training data that's totally skewed toward academic writing from the last couple decades, so anyone with a slightly formal style gets flagged. and honestly, if a teacher is gonna use these things, they should at least know what the tool is actually detecting. because right now it's just a black box that makes people paranoid and creates more problems than it solves.