Bot Problems
Aug. 2nd, 2025 12:22 am![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
soc_puppetI briefly saw (and participated in, for exactly one reblog) a thread on Tumblr about whether AO3 should allow works produced by Generative AI. OP was in favor, as was the first reblogger, where as at least one reblogger was firmly against all AI generated works on AO3, and several (myself included) shared the reasons why AI generated works currently are allowed on AO3. One of the rebloggers who is firmly against any AI generated works being allowed on AO3 said, "Like anything else in a shared social space, an enforced social norm requires actual social consequences for breaking it. This? Is just capitulation."
Anyway, that person also said that they're done talking about it, and I wanted to respect that, so I'm taking my thoughts over here. If you recognize the quote in question, or know who the person is, I will politely ask you to respect their boundaries and not go looking for the post and/or reblogging it from them with your own commentary; we can talk in the comments of this post, which they can safely ignore. I'm personally struggling with the point they raised, and will probably not be able to get to sleep without working some of it out, so I'm doing so over here where, again, they should be able to safely ignore it.
So. AI generated works on AO3.
For moral and ethical reasons, AO3 should not host AI generated works. This I fully agree with. For the same reasons why LLMs should not be used in place of human creativity (destroying the environment at a much faster rate than general electronic use, plagiarism, etc etc), AO3 should not give these works a safe home.
Unfortunately, that leaves us with the practical questions related to How to enforce a Gen AI works ban?
The biggest practical question is, "How can we tell something is AI generated?" To the best of my knowledge, outside of people admitting that they generated a work using ChatGPT and the like, there is no reliable way.
With many LLMs trained on AO3 itself, legalities aside, that means that a lot of the commonly touted "tells" come from AO3's "house style". So using those "easy tells" is out.
On top of that, other "AI detectors" will get stuck on stylistic differences that actually indicate someone is a non-native English speaker; AO3 already has a demonstrated racism problem, and even if we assume that everyone reporting fics for being AI generated is doing so because they actually believe the fic is AI generated (having run it through one of those AI detectors, for example), a lot of fics that shouldn't be are going to end up targeted.
And speaking of reporting in good faith, let's talk about reporting in bad faith. There are plenty of anti-fans who want AO3 to stop hosting much of the very content it was built for; some of them are hopefully getting a clue over the Itch.io payment processor thing, but that will never get through to all of them. (Hell, some of them might even be behind the Itch.io payment processor thing!) There are also already reports of (potentially bot-generated) comments on fics accusing them of being AI generated, which is not helping the morale of any writers. I can all too easily picture malicious actors going around and reporting any fic they don't like as being AI generated. What a waste of time for everyone involved! And with the Policy and Abuse team already poorly trained and overworked, adding this to their plate just seems... cruel, almost.
Practically speaking, this also means that the only works that would get taken down are those whose posters admit are generated by AI. And, given AO3's stance on not addressing off-site harassment, it would possibly be only works whose creators admit on AO3 itself are AI generated; I don't know how AO3 deals with any off-site plagiarism admissions, which is what I'd compare it to, so I'm at a loss there.
Which brings us to the big argument AO3 is using in favor of hosting AI generated works: At least when AI generated works are allowed, people will admit that's what they are, and people who don't like that can filter them out or just plain not read.
I honestly feel like this is potentially the weakest argument of the bunch, and possibly the only one I have an actual solution for; if AO3 disallows AI generated works, then yes, people would still post them without admitting that they're AI generated. Hell, they might even still admit to it! Look at how many people get caught sharing Ko-Fi and Patreon links in their notes! At least that's a report-able offense.
But right now I'm stuck on the question: Are the people who use Gen AI to "write" fics admitting it? Or is Gen AI unpopular enough that they've largely stopped being honest about it? If it's the latter, AO3 may as well roll out a "No AI Generated Content" rule, with the clause that any reporting must have evidence of self-admitted use of AI. Otherwise, we're basically already living in the universe where we can't tell for sure whether any content is AI generated, just without any recourse if anyone who does post it slips up and admits it.
Anyway, those are my current thoughts. So now probably the thing that will keep me up is trying to decide whether to create a Tumblr poll about that last point ("Do you post AI generated works to AO3, and if so, do you admit that they're AI generated?")
Anyway, that person also said that they're done talking about it, and I wanted to respect that, so I'm taking my thoughts over here. If you recognize the quote in question, or know who the person is, I will politely ask you to respect their boundaries and not go looking for the post and/or reblogging it from them with your own commentary; we can talk in the comments of this post, which they can safely ignore. I'm personally struggling with the point they raised, and will probably not be able to get to sleep without working some of it out, so I'm doing so over here where, again, they should be able to safely ignore it.
So. AI generated works on AO3.
For moral and ethical reasons, AO3 should not host AI generated works. This I fully agree with. For the same reasons why LLMs should not be used in place of human creativity (destroying the environment at a much faster rate than general electronic use, plagiarism, etc etc), AO3 should not give these works a safe home.
Unfortunately, that leaves us with the practical questions related to How to enforce a Gen AI works ban?
The biggest practical question is, "How can we tell something is AI generated?" To the best of my knowledge, outside of people admitting that they generated a work using ChatGPT and the like, there is no reliable way.
With many LLMs trained on AO3 itself, legalities aside, that means that a lot of the commonly touted "tells" come from AO3's "house style". So using those "easy tells" is out.
On top of that, other "AI detectors" will get stuck on stylistic differences that actually indicate someone is a non-native English speaker; AO3 already has a demonstrated racism problem, and even if we assume that everyone reporting fics for being AI generated is doing so because they actually believe the fic is AI generated (having run it through one of those AI detectors, for example), a lot of fics that shouldn't be are going to end up targeted.
And speaking of reporting in good faith, let's talk about reporting in bad faith. There are plenty of anti-fans who want AO3 to stop hosting much of the very content it was built for; some of them are hopefully getting a clue over the Itch.io payment processor thing, but that will never get through to all of them. (Hell, some of them might even be behind the Itch.io payment processor thing!) There are also already reports of (potentially bot-generated) comments on fics accusing them of being AI generated, which is not helping the morale of any writers. I can all too easily picture malicious actors going around and reporting any fic they don't like as being AI generated. What a waste of time for everyone involved! And with the Policy and Abuse team already poorly trained and overworked, adding this to their plate just seems... cruel, almost.
Practically speaking, this also means that the only works that would get taken down are those whose posters admit are generated by AI. And, given AO3's stance on not addressing off-site harassment, it would possibly be only works whose creators admit on AO3 itself are AI generated; I don't know how AO3 deals with any off-site plagiarism admissions, which is what I'd compare it to, so I'm at a loss there.
Which brings us to the big argument AO3 is using in favor of hosting AI generated works: At least when AI generated works are allowed, people will admit that's what they are, and people who don't like that can filter them out or just plain not read.
I honestly feel like this is potentially the weakest argument of the bunch, and possibly the only one I have an actual solution for; if AO3 disallows AI generated works, then yes, people would still post them without admitting that they're AI generated. Hell, they might even still admit to it! Look at how many people get caught sharing Ko-Fi and Patreon links in their notes! At least that's a report-able offense.
But right now I'm stuck on the question: Are the people who use Gen AI to "write" fics admitting it? Or is Gen AI unpopular enough that they've largely stopped being honest about it? If it's the latter, AO3 may as well roll out a "No AI Generated Content" rule, with the clause that any reporting must have evidence of self-admitted use of AI. Otherwise, we're basically already living in the universe where we can't tell for sure whether any content is AI generated, just without any recourse if anyone who does post it slips up and admits it.
Anyway, those are my current thoughts. So now probably the thing that will keep me up is trying to decide whether to create a Tumblr poll about that last point ("Do you post AI generated works to AO3, and if so, do you admit that they're AI generated?")
no subject
Date: 2025-08-03 04:13 am (UTC)Drives me NUTS to see all these smart people actively make themselves dumber on purpose.
But I digress :) All that to say, this is a very important conversation, and I'm glad to see people talking about it. I'm no longer on tumblr myself, but I'd love to hear the results if you put out a poll on this.
no subject
Date: 2025-08-06 06:12 pm (UTC)Also I genuinely enjoy writing stories myself. I can't draw very well, but I love doodling whatever I doodle myself. I'm sure if they made an AI robot that knitted or crafted some people would externalize their hobbies to it. Personally, I don't understand the need to. I miss the time when robot labor was seen as a way to give people more free time, not steal away their creativity, but maybe I'm just old.
no subject
Date: 2025-08-07 10:36 pm (UTC)I feel older every day, with this stuff.