Technology

The Thélyson Orélien controversy raises new questions about whether detection software can be trusted

A lot of people here might already be aware of how AI checkers are unreliable at best, and more often just terrible. What I learned from this article is that they might actually harm some groups of people more than others.

NOTE: The original title of this article was "Bestselling Novelist Dropped from Major Prize over Suspected AI Use". I added the extra context because people might not read the body of the post.

Here are some excerpts that I thought were relevant, with bold emphasis applied by me. Please check out the full article if you want more context outside of what I am highlighting here:

This time, the controversy surrounds C’était ça ou mourir, the first novel by Haitian Canadian author Thélyson Orélien, which has already become a huge success in France. On September 26, the prestigious Académie Goncourt announced it was dropping the title from the longlist for its eponymous literary prize because the prize administrators had concluded it was written with the help of AI. The AI-detection software Pangram determined there was a 100 percent chance the novel was at least in part AI generated.

The author denies using AI, and his French publisher, Grasset, has come out strongly in his defence. Grasset said in a statement that it “strongly objected” to the allegations, which it characterized as “malicious.” The Globe and Mail quoted the French publisher as calling the accusations against Orélien “fake news.” His original French-language Canadian publisher, by contrast, has taken a more measured response to the allegations. The Quebec publisher, Les Éditions du Boréal, told the New York Times, “[A]t this stage, it is impossible for us to confirm or deny the allegations the author is facing.” They went on to say that it is important “not to draw hasty conclusions.”

This is not the first time an award-nominated author has faced accusations of using AI to create written work. Earlier this year, the Commonwealth Short Story Prize encountered criticism after Jamir Nazir’s regional winner, “The Serpent in the Grove,” was accused by social media users of having been written with AI. After a review that included interviews with all the regional winners and examinations of early drafts, edits, and time-stamped emails, the organization that runs the award decided there was no basis to believe Nazir used AI in his writing. The story went on to win the overall Commonwealth Short Story Prize.

The issue of false positives is not incidental: somewhere around 90 percent of the texts used to train large language models are in English, so the French language would throw up difficulties for AI checkers, which themselves use AI to make their determinations. In addition, Orélien is Haitian by background and employs a style of writing heavily influenced by Caribbean oral tradition, which relies on repetition and other tropes often cited as determinants of AI assistance. “I come from a Haitian and Caribbean literary tradition in which poetry, imagery, repetition, and rhythm carry a living voice,” he said in a statement quoted by the CBC. “One might debate my style, but treating its techniques as evidence of artificial intelligence has nothing to do with literary criticism.”

Sutherland, who, like Orélien, is Black, made a similar observation to the Globe about her award-winning story. “This story is rooted in Vincentian oral storytelling tradition, which by nature is formulaic and relies on repetition, rhythm, and patterned language as core elements of their structure,” she said.

The biases and blind spots in AI-detection software are only one issue. Equally troubling is the notion of using AI to detect AI use, an apparent tautology that doesn’t have many safeguards or levels of verifiability. Then there are the people using the software themselves. The accusations against Orélien arose via an anonymous X account run in part by Samuel Fitoussi, a journalist at the right-wing newspaper Le Figaro. His stated reason for using the account to publicize his accusation, according to the New York Times, is that “he thought an anonymous account would be trusted more than his column in a right-leaning newspaper.” Fitoussi may be correct about that, if only because the French right wing has become increasingly antipathetic to immigrant communities in the country.