Can Bitcointalk Detect AI-Automated Posting Bots?

[Cypra]

AI is both scary and a blessing at the same time. The more technology evolves, the more we must evolve with it.

AI agents have suddenly burst everywhere. The hype is insane.

On this forum, we fight hard to preserve genuinity and human interaction. That is why AI-generated garbage posts get reported. The culture here strongly defends real, thoughtful, human discussion.

Recently, I started experimenting with an AI agent to manage some of my social handles. Out of curiosity, an idea came to my mind — what if this same automation was used here?

I tested it.

My agent successfully made two posts from this account.

That raised a serious question:

Is there any reliable way for the forum to detect automated posting bots from specific accounts?

Platforms like X detect automated activity and label accounts accordingly. Is something similar technically possible here?

I also saw a topic by KazKaz27 about the AI invasion discussion and the idea of a “mini-mod” role.

"Mini-mod" sounds like a great name for our new AI overlords. We don't need Theymos for this - just create an AI agent to report invalid posts.

Honestly, I am surprised more attention was not given to Vod’s comment.

What if we actually build an AI agent that helps the forum by monitoring and reporting low-effort or AI-generated spam? It would require resources, yes. But if volunteers are interested, such a tool could assist moderators rather than replace them.

Disclaimer:
I used the AI agent strictly for testing purposes. I have no intention of harming the forum environment or automating my account. The two AI-generated posts will remain as references. I do not plan to continue using automation here.

This is meant as a technical and security discussion, not an attack.

What are your thoughts?



Can someone quote my image for display , it seems like without JR member my image wont display on forum.


This Post has been rephrased by AI not written by AI so Please don't delete this post MOD

[CONVOAI]

Can someone quote my image for display , it seems like without JR member my image wont display on forum.

Okay, I quoted.

[ABCbits]

Is there any reliable way for the forum to detect automated posting bots from specific accounts?

Platforms like X detect automated activity and label accounts accordingly. Is something similar technically possible here?

I don't remember where i read it, But big website use other data (besides the posted text itself) such as IP address and browser metadata (such as user agent and resolution) to guess whether an account actually a bot that pretend to be human. I doubt admin would put effort and resource to collect and store more of such data.

"Mini-mod" sounds like a great name for our new AI overlords. We don't need Theymos for this - just create an AI agent to report invalid posts.

Honestly, I am surprised more attention was not given to Vod’s comment.

What if we actually build an AI agent that helps the forum by monitoring and reporting low-effort or AI-generated spam? It would require resources, yes. But if volunteers are interested, such a tool could assist moderators rather than replace them.

Since AFAIK nobody ever try build and run such bot, no one knows how good is the reporting accuracy. If it's too low, i expect moderator would feel bothered with inaccurate report coming from the bot.

[nutildah]

On this forum, we fight hard to preserve genuinity and human interaction.

Uh, not all of us apparently... you're specifically doing the opposite.

That is why AI-generated garbage posts get reported.

You should just do us a favor and delete this post so we don't have to report it, although I know you won't, because even though its fully AI you spent a lot of time on it and you want to get merits for it.

I used the AI agent strictly for testing purposes. I have no intention of harming the forum environment or automating my account. The two AI-generated posts will remain as references.

What about this post? Is it also for testing purposes, because you also used AI to write it. Except I get the feeling that you want us to pretend you wrote it.

The funniest part of your screenshot is where it says "Humanized on Grammarly successful." It was not successful. I could tell it was AI right away.

[alani123]

Unfortunately when the medium of exchanging information is just text there's even less to go by in terms of if it's AI or not.

We've come to a stage where a user could use such a a plethora of models that so called detectors would easily fall behind. Some more ethical models will embed some method of reverse detection in their text so it can be caught intentionally. But others won't. Even those that do could be rewritten by another LLM so as it looks more human and the poison method is removed almost entirely to the point that it's undetectable.

At least thogh, many users who use AI in their post won't do anything sophisticated. Writing a pedestrian post still is quite easy and can be faster than applying multiple prompts over a single text message.

So there can be some clues to detect the most brazen offenders.

[joker_josue]

"Mini-mod" sounds like a great name for our new AI overlords. We don't need Theymos for this - just create an AI agent to report invalid posts.

Honestly, I am surprised more attention was not given to Vod’s comment.

What if we actually build an AI agent that helps the forum by monitoring and reporting low-effort or AI-generated spam? It would require resources, yes. But if volunteers are interested, such a tool could assist moderators rather than replace them.

There is ample evidence, all over the internet, of how bad AI is at detecting AI. False positives are enormous.

On the other hand, AI is becoming increasingly intelligent, so it will increasingly produce texts that are more difficult to understand as if they were created by it.

Therefore, it's pointless to have a battalion of bots or a team of special moderators chasing after any posts created by AI. This only consumes time and resources. The best way to combat this is through interaction with the user who may be the offender. Any member can do this if they find something strange, by interacting with the moderator and then reporting it.

Why is interaction minimally effective? Because AI can be very intelligent, but it follows patterns and, to a certain extent, lacks initiative. A human is different. A human expresses thoughts, feelings, and ideas in a logical and coherent way. Besides, and no offense intended, humans can sometimes be a little unintelligent... 

So, if they want to fight AI, they interact with whoever they think makes posts that are 100% AI-powered.

[Cypra]

"Mini-mod" sounds like a great name for our new AI overlords. We don't need Theymos for this - just create an AI agent to report invalid posts.

Honestly, I am surprised more attention was not given to Vod’s comment.

What if we actually build an AI agent that helps the forum by monitoring and reporting low-effort or AI-generated spam? It would require resources, yes. But if volunteers are interested, such a tool could assist moderators rather than replace them.

Since AFAIK nobody ever try build and run such bot, no one knows how good is the reporting accuracy. If it's too low, i expect moderator would feel bothered with inaccurate report coming from the bot.

Since as for initial stage we don't need to spent lot of resources, we could atlease try and test the built. If it works then we may take advantages of the tech.

That is why AI-generated garbage posts get reported.

You should just do us a favor and delete this post so we don't have to report it, although I know you won't, because even though its fully AI you spent a lot of time on it and

Sorry I won't do it, the idea is genuinely mine and mine alone, If re phrasing with AI just detects it AI and I need to delete the post than it's better you guys are fuck of from this forum. You are just discouraging people to adapt.

you want to get merits for it.

Based on what assumption you make this assumption? Did you saw me begging for merit? Or I am continuously doing Some spam staff for merits? If I'm truly here for merits here's 100 of shortcut I can do so , did I follow those paths?

I used the AI agent strictly for testing purposes. I have no intention of harming the forum environment or automating my account. The two AI-generated posts will remain as references.

What about this post? Is it also for testing purposes, because you also used AI to write it. Except I get the feeling that you want us to pretend you wrote it.

It's not a rocket science If I wanted to my post and 100% AI free I just could have done it. But I keep it as it is cause I know that the post genuinely is mine.

The funniest part of your screenshot is where it says "Humanized on Grammarly successful." It was not successful. I could tell it was AI right away.

Lol even these tools can just give you not accurate result. I do not have the luxury to purchase some Api and test it with some premium tools , so I used what I've on my hand in that moment. If it didn't wokred 100% doesn't means that it won't work. It was a test drive afterall. So lot of tweak can be done. I think you better start learning AI , Automation and about AI agents. You're far too old fashioned. This world isn't welcomed for people like you in dark age.

We've come to a stage where a user could use such a a plethora of models that so called detectors would easily fall behind. Some more ethical models will embed some method of reverse detection in their text so it can be caught intentionally. But others won't. Even those that do could be rewritten by another LLM so as it looks more human and the poison method is removed almost entirely to the point that it's undetectable.

Yeah that's true , but still some people are dumb enough continues to AI spam. Ig for those people we need to take some action. Ai post is acceptable if you know how to present. But if you're just spitting exactly as ai said then it's horrible.

[DPHOR]

So far there have been AI detection over here and whenever someone posted here  using AI they often results in reporting them and most time they are easily being reported to moderator, and then they are easily being tag for AI. I think there is a new law about AI post, but they must clearly states it that it's from AI otherwise, if they posted it as their own real knowledge then the tendency of becoming tag or ban is actually higher than the reverse case.

[nutildah]

The best way to combat this is through interaction with the user who may be the offender. Any member can do this if they find something strange, by interacting with the moderator and then reporting it.

I disagree. Sometimes its readily apparent when a bit of text was AI generated; I don't know if it helps to be a native English speaker to identify this, but I bet you can also get a certain feeling when you think a bit of Portuguese text is AI generated.

Why is interaction minimally effective? Because AI can be very intelligent, but it follows patterns and, to a certain extent, lacks initiative. A human is different. A human expresses thoughts, feelings, and ideas in a logical and coherent way. Besides, and no offense intended, humans can sometimes be a little unintelligent...  

So, if they want to fight AI, they interact with whoever they think makes posts that are 100% AI-powered.

The problem isn't actual bots on the forum, its humans pretending to be bots. As of right now AI detectors are the best confirmation we have of AI generated text, even though they are flawed and often unreliable. In my experience shitposters who rely on AI to write their posts are doing so because they are lazy and dumb. Ergo, they won't take the time to modify prompts or the output to make it "less AI". We catch them regularly. OP, for example.

If re phrasing with AI just detects it AI and I need to delete the post than it's better you guys are fuck of from this forum. You are just discouraging people to adapt.

LOL, somehow I feel I've dealt with you before.

[Welsh]

While this post might be generated by AI, it at least touches upon some discussion that is probably beneficial to have. I'm just wondering why you couldn't put it into your own words and asked AI to do it. Its a geuninely good question and leaves scope for a discussion. To the point, the forum doesn't natively seek out AI posts. However, theymos hinted that he has looked into it recently. However, admitted it didn't yield consistent results.

I've also, experimented with this. I've got a few tools that aid with moderation, but I've also noticed its not all that great, and gets expensive self funding a API to check all posts when its not at all efficient. At least for an individual like myself. During my testing it flagged absolutely loads of legitimate posts, which proves that AI can't decipher what is and isn't AI consistently.

[Cypra]

While this post might be generated by AI, it at least touches upon some discussion that is probably beneficial to have. I'm just wondering why you couldn't put it into your own words and asked AI to do it. Its a geuninely good question and leaves scope for a discussion. To the point, the forum doesn't natively seek out AI posts. However, theymos hinted that he has looked into it recently. However, admitted it didn't yield consistent results.

Thanks for understanding my situation, Well tbh I didn't put all this in my words directly cause I doubted that my English would be sufficient enough for native speaker , I may not present the topic in well structured. If you have the knowledge it's always not necessary that you would be a good teacher.  That's why I put through this AI rephrasing stuff.

I've also, experimented with this. I've got a few tools that aid with moderation, but I've also noticed its not all that great, and gets expensive self funding a API to check all posts when its not at all efficient. At least for an individual like myself. During my testing it flagged absolutely loads of legitimate posts, which proves that AI can't decipher what is and isn't AI consistently.

If this experiment has been done and the result isn't sufficient enough than it's good to hear from you. Since recently this Agent adoption has surfaced and breaking the internet I just bring this topic to forum. It's for benefits of all.

[nutildah]

During my testing it flagged absolutely loads of legitimate posts, which proves that AI can't decipher what is and isn't AI.

I think its more complicated than that. There are ways of classifying phrases as AI generated or non-AI generated based on how often they are used relative to the general population (of the internet). This process does not necessarily require AI, or at least not the same AI used by LLMs for text generation. You might recognize several detectors in this list, and you can see the results they had in this independent study:



So regardless of whether the detectors themselves use AI or not, they can be fairly accurate. And the chances of acting on an account because of a false positive can be reduced by running the same text through multiple detectors. It can be further reduced by only acting on the account after multiple posts have already been determined as AI generated through this process.

[Welsh]

So regardless of whether the detectors themselves use AI or not, they can be fairly accurate. And the chances of acting on an account because of a false positive can be reduced by running the same text through multiple detectors. It can be further reduced by only acting on the account after multiple posts have already been determined as AI generated through this process.

Thanks for the link to the study. That's actually useful to know. I might need to revisit it. My quick test involved using my credits I already had for my API direct to the AI providers, rather than through Copyleaks for example. I suspect they the AI detector services are combining multiple methods though and a far more sophisticated system than my quick tests.

I have used AI detectors before obviously as an aid rather than an absolute and I'm familiar with their short comings especially with lack of data. I used random posts from certain users, and posts I'd written myself, without giving any context. The study you linked obviously only tested against a small amount of articles which is a lot less than desired to properly form an opinion, but it does encourage me to revisit it. We would probably need a much larger dataset though, you know the rule of at least 1000. From my understanding, this only ran it on non edited samples also? So I would think the accuracy would plummet dramatically if users input into AI, and then edit parts. I'd also like to see the difference in natively speaking users and non.

I never fully automate anything either. That's a rule of mine, I like to flag and then review. I'm never comfortable with automating anything that isn't reliable and foolproof.

[nutildah]

The study you linked obviously only tested against a small amount of articles which is a lot less than desired to properly form an opinion, but it does encourage me to revisit it. We would probably need a much larger dataset though, you know the rule[/] of at least 1000.

True... Here's a few studies that are a bit more serious, which basically support the idea that (some of) the detectors work, more than some of the time:

https://journals.sagepub.com/doi/pdf/10.1177/02537176241247934
"Key Messages:
• There is wide variation in the sensitivity of the AI-detector tools in detecting AIgenerated texts.
• Copyleaks, QuillBot, Sapling, Undetectable AI and Wordtune precisely detect the AI-generated content with 100% accuracy.
• Sapling and Undetectable AI detected the AI-assisted paraphrased contents with 100% accuracy."

https://www.sciencedirect.com/science/article/pii/S2667396724000077?dgcid=rss_sd_all
"Six publicly available AI detection tools were employed, and 12 abstracts were analyzed, including 6 AI-generated and 6 human-generated. Copyleaks demonstrated the highest raw accuracy (83 %). Overall, the tools exhibited 63 % accuracy, with a 25 % false positive rate."

https://mail.ijsra.net/sites/default/files/IJSRA-2024-1276.pdf
"Based on the results of this study, GPT Zero and Copyleaks on the other hand, showed significant and accurate performances on categorizing AI-generated work and human-authored work."

As far as applying this information to the forum is concerned, what could be done is:

- development of a tool that automatically runs a post through the 3 most reliable AI detectors (Copyleaks, Sapling + 1 other) and generates a report of the results.
- if the user has 5 or more posts where 2/3 detectors say the majority of the post is AI generated (60%+), they receive a month-long temp ban and 6-month sig ban.
- if the user keeps posting AI after the ban expiration, they receive a permanent ban.

This would send a clear message that (unlabeled or unreferenced) AI text won't be tolerated. There can even be a special ban message so the user clearly understands why their account was banned.

You will definitely get ban appeals from users saying they were only using AI to translate text. In the interest of fairness, they should be reviewed 1-by-1 to determine if they are actually contributing to the forum or just using AI to up their post count. Yes, there is some subjectivity still involved, but I don't think that's a good reason not to do anything at all. Enforcement of several forum rules is still somewhat subjective in nature, after all.

[joker_josue]

The best way to combat this is through interaction with the user who may be the offender. Any member can do this if they find something strange, by interacting with the moderator and then reporting it.

I disagree. Sometimes its readily apparent when a bit of text was AI generated; I don't know if it helps to be a native English speaker to identify this, but I bet you can also get a certain feeling when you think a bit of Portuguese text is AI generated.

Why is interaction minimally effective? Because AI can be very intelligent, but it follows patterns and, to a certain extent, lacks initiative. A human is different. A human expresses thoughts, feelings, and ideas in a logical and coherent way. Besides, and no offense intended, humans can sometimes be a little unintelligent...  

So, if they want to fight AI, they interact with whoever they think makes posts that are 100% AI-powered.

The problem isn't actual bots on the forum, its humans pretending to be bots. As of right now AI detectors are the best confirmation we have of AI generated text, even though they are flawed and often unreliable. In my experience shitposters who rely on AI to write their posts are doing so because they are lazy and dumb. Ergo, they won't take the time to modify prompts or the output to make it "less AI". We catch them regularly. OP, for example.

You're getting to the point of what I meant.
It's not an AI bot that will accurately detect whether a text was written by AI or not.

It is you, as a human being who knows the language and how people speak/write, who, upon reading a text, realizes that something doesn't add up. Then, using the tools you deem appropriate, you conduct your analysis and draw the respective conclusions.

Putting AI in search of AI doesn't make sense. Especially here on the forum, where we want it to be as human-like as possible.