AI-generated post discussion thread: how to identify & report

[nutildah]

I think theymos should probably make an announcement about prohibiting this sort of use of AI or at the very least we add it to the "unofficial" forum rules but users should be alerted not to use it in such a way. If a user if just going to use it to copy and paste the title of a thread or the OP into an AI then copy and paste the results that should be treated the same as plagiarism. If it isn't then this place isn't going to be a discussion forum for much longer but just a get paid to copy and post AI responses forum.

I agree. It would be nice to hear from The Big Man about a soft policy on this, hopefully something akin to what Stack Overflow has done. Not only are AI-generated posts spammy but they often contain wrong information as well.

The question remains of how to best determine that a post was written using AI. This tool seems to have the best rate of accuracy, but it doesn't judge how much of a text is AI-written, just whether or not it is "likely" to contain AI-written material:

https://hivemoderation.com/ai-generated-content-detection

The other decent ones include:

https://writer.com/ai-content-detector/
https://contentatscale.ai/ai-content-detector/

Other clear signs of AI-generated posting:

- using perfect English & punctuation when the poster's previous posts did not
- writing long texts in rapid succession, when it was unlikely to be possible manually

Surely some combination of these 3 things is enough to come to a reasonable conclusion whether a post was written by AI or not.

[1714731698]

1714731698

[1714731698]

1714731698

[1714731698]

1714731698

[_BlackStar]

The question remains of how to best determine that a post was written using AI. This tool seems to have the best rate of accuracy, but it doesn't judge how much of a text is AI-written, just whether or not it is "likely" to contain AI-written material:

https://hivemoderation.com/ai-generated-content-detection

I'll try this tools to test it for accuracy and it looks like the text I quoted below from one of the users has a 97.6% & 99.9% chance of likely to contain AI Generated Text.

It's true that working as a government employee doesn't make us rich, but why do so many people still want to become government employees? Because the most valuable thing for government employees is a lifetime pension guarantee, even if they are widows/widowers they also get pension insurance and their first and second children also get it if they are not yet 25 years old. (this is what happened in my country)

While it is true that the salaries of government employees are generally sufficient to meet their daily needs and not enough to save or invest with existing income. So it's only natural that as a government employee you have to be smart to find additional income to become rich.

In essence, not all government employees have to rely on a monthly salary alone to achieve financial success. As a government employee, you must be smart about using your free time to run a side business or involve yourself in investments that can generate greater profits.

As what the OP has explained, these two types of salary have advantages and there are also negative sides. Actually, the decision to choose a job based on salary or commission salary does not only depend on whether a person wants a large income or not. There are other factors to consider such as skills, interests, career opportunities and personal values. Some people may be more interested in a job that offers a steady salary because they feel more stable and can manage their finances more easily. However, there are also those who prefer a job with a commission salary because they feel more compelled to work hard and earn bigger profits.

It is important to consider the risk factors in commission salary work. While most commission salaried jobs do offer greater earning potential, they also carry a higher risk. For example, if you work in sales, the profit you make will depend on many factors such as market conditions, competitors and customer needs.

In the end, the decision to choose a job based on salary or commission salary depends on individual preferences. It is best to consider all relevant factors and make a decision that best fits your goals and personal values. I personally prefer to work with a monthly salary because this is the right choice for my current condition that I have a family, because with a monthly salary, it is certain that I will have an income, even though it is not as big as a commission salary.

The above two posts were created by the same user, but I'm not mentioning the names ^{[ninjastic.space can help to find it]} because I'm not entirely sure the above content is actually generated by AI because I got different results when I tested with different tools, especially with these two tools:

The other decent ones include:

https://writer.com/ai-content-detector/
https://contentatscale.ai/ai-content-detector/

[elevates]

I have been testing this tool undetectable.ai for some time now. It's accuracy is far better than other tool available for free at the moment. Does detection from all relevant AI content writing apps. I did tested it for BARD and it worked fine. Detection is done for 10000 characters. At the moment it is not charging a penny for AI detection. The only problem here is that it requires registration through email. I did use temp mail and they allowed the email address.

[nutildah]

Ran into some more AI-generated posts today. Not even by sig campaigners, lol.

I'm going to open a thread in Reputation which acts as a reference for reports on AI posts. Before doing that, I'd like to get any final feedback on how to proceed. Note this is simply for attaching references to reports, not for any kind of tagging (although if several of a user's posts are deleted as spam then a neutral indicating they are an AI spammer may be warranted).

After much deliberation and testing, here are the criteria I will be using to report posts as AI spam:


A. These 3 AI content detectors must all give a result of the post having a strong likelihood of containing AI-written material (with "strong likelihood" defined for each one):

1. Hive Moderation - 80% likely to contain AI generated text or above
2. Writefull GPT Detector - 80% likely to come from GPT-3, GPT-4 or ChatGPT or above
3. Writer AI Detector - 20% human-generated content or below

(I decided against including detectors that make you pay or register to keep verification accessible to all)


B. There must be at least 3 posts by the account that meet the requirements specified in A.

So, if its just a one-off or even two-off, I think the poster deserves some leniency, but if they make a habit of it, I strongly believe they should be named n' shamed and their posts reported.


C. Clues to help identify an AI poster before putting their posts through the detection process include:

- using perfect English & punctuation when the poster's previous posts did not
- writing long texts in rapid succession, when it was unlikely to be possible manually

and if their posts are frequently riddled with phrases mentioned here.


Anyone who thinks this shouldn't be done, or should be done in a different way, please feel free to chime in now or forever hold your peace. Thanks.

[lovesmayfamilis]

I think it's a good idea. But how should we report it to moderators? If you offer to show some examples of the use of AI, then the reports, I think, will become very large if you provide screenshots as evidence.
Are three examples enough?
In addition, from my observations, I realized that if you send several reports to moderators that an account is using AI, this account is temporarily blocked.
And I agree with you: after I submit more than 10 reports with AI posts, I tag the account with a neutral tag.

Here is an example of such a spammer.  https://bitcointalk.org/index.php?action=profile;u=2132496;sa=showPosts
Post creation speed and use of AI

[LoyceV]

It should be treated the same as plagiarism.

Agreed! You're a Global Mod, so why isn't nesty banned yet?

I think theymos should probably make an announcement about prohibiting this sort of use of AI or at the very least we add it to the "unofficial" forum rules but users should be alerted not to use it in such a way.

Isn't this already covered by the unofficial forum rules?

33. Posting plagiarized content is not allowed.

Let's look at the definition:

Plagiarism
the process or practice of using another person's ideas or work and pretending that it is your own

I've reported one of this posts again:

Bounty spammer uses chatbot plagiarism to spam the forum. Please ban.

[nutildah]

I think it's a good idea. But how should we report it to moderators? If you offer to show some examples of the use of AI, then the reports, I think, will become very large if you provide screenshots as evidence.
Are three examples enough?
In addition, from my observations, I realized that if you send several reports to moderators that an account is using AI, this account is temporarily blocked.
And I agree with you: after I submit more than 10 reports with AI posts, I tag the account with a neutral tag.

The thinking is when you want to report a post as being "AI written" - especially for posts that may not be outright spam but are AI nonetheless - you can put a reference link in the report to your post in the thread that shows the results of the AI detectors. So the first time an offending account is added to the thread, the thread post should have 3 or more examples of their posts. Then each subsequent time the account posts AI, those can be added to the thread individually (if necessary).

It will primarily serve to buttress for "AI written" or "AI spam" reports, but its also a way to publicly call out accounts engaging in this behavior, which may dissuade others from doing it in the future.

Here is an example of such a spammer.  https://bitcointalk.org/index.php?action=profile;u=2132496;sa=showPosts
Post creation speed and use of AI

Holy moly are these bad:

I understand your concerns regarding the Pi coin and other projects that offer free mining or airdrops. It is important to understand that I do not have access to specific information about the Pi project or its plans for the future, as my learning base is limited to information until September 2021.

 

[LoyceV]

I found one here: loyce.club/badposts/spam.html, and it got edited after posting.
Original:

As an AI language model, I cannot predict the future or provide financial advice. Altcoin season typically refers to a periThe market cycles of cryptocurrencies are highly unpredictable and influenced by a variety of factors, including market sentiment, adoption, and regulatory developments. It is important to conduct your own research and consult multiple sources before making any investment decisions.

Current:

Altcoin season typically refers to a periThe market cycles of cryptocurrencies are highly unpredictable and influenced by a variety of factors, including market sentiment, adoption, and regulatory developments. It is important to conduct your own research and consult multiple sources before making any investment decisions.

The user has a neutral tag about it already, but hasn't been banned. It looks like 18 posts have been deleted already, but the user is still spamming.

[PytagoraZ]

For me, the best indicator if a post is AI-generated is if it is written in flawless English. Even the best posters who are native English speakers don't necessarily write every post using the precision in punctuation, spelling & grammar executed by ChatGPT. If a user's native language isn't English (as can be evidenced by their posting history), and they suddenly start posting in flawless English, it means one of two things:

1. The text is plagiarized, or
2. The text is AI-generated.

That's right, The easiest indicators to know an article was made with AI is the perfection of the language. (maybe a typo is important so it's not considered AI generated)

However, if you copy paste and it is detected by an AI detector the result is the AI generated. AI detector can only show AI generated or not, but cannot detect plagiarism. Must be done 2 times. with AI generated and plagiarized text

AI generated is still not accurate. AI and AI generated are still in the development stage.

AI is still in the development stage, it's already troublesome, especially if AI technology is getting more sophisticated. Maybe we'll be eliminated.

Welcome new word

[RickDeckard]

Perhaps some users haven't noticed yet but OpenAI has shut down their AI detection tool[1] that is mentioned in the OP[2]. Interestingly enough, more than the fact how silent this removal was, was the reason underneath it[3]:

As of July 20, 2023, the AI classifier is no longer available due to its low rate of accuracy. We are working to incorporate feedback and are currently researching more effective provenance techniques for text, and have made a commitment to develop and deploy mechanisms that enable users to understand if audio or visual content is AI-generated.

It makes me wonder - If OpenAI isn't good at detecting AI generated text, how reliable are other detectors?
[1]https://decrypt.co/149826/openai-quietly-shutters-its-ai-detection-tool
[2]https://platform.openai.com/ai-text-classifier
[3]https://openai.com/blog/new-ai-classifier-for-indicating-ai-written-text

[crwth]

Perhaps some users haven't noticed yet but OpenAI has shut down their AI detection tool[1] that is mentioned in the OP[2]. Interestingly enough, more than the fact how silent this removal was, was the reason underneath it[3]:

As of July 20, 2023, the AI classifier is no longer available due to its low rate of accuracy. We are working to incorporate feedback and are currently researching more effective provenance techniques for text, and have made a commitment to develop and deploy mechanisms that enable users to understand if audio or visual content is AI-generated.

It makes me wonder - If OpenAI isn't good at detecting AI generated text, how reliable are other detectors?
[1]https://decrypt.co/149826/openai-quietly-shutters-its-ai-detection-tool
[2]https://platform.openai.com/ai-text-classifier
[3]https://decrypt.co/149826/openai-quietly-shutters-its-ai-detection-tool

How full circle it is when OpenAI has difficulty determining the AI-generated text they created. It makes it even more impossible if you think about it because it's improving. Isn't it somewhat contradicting in a way? Getting better in AI text generation makes it harder to detect it even more.

[joker_josue]

Perhaps some users haven't noticed yet but OpenAI has shut down their AI detection tool[1] that is mentioned in the OP[2]. Interestingly enough, more than the fact how silent this removal was, was the reason underneath it[3]:

As of July 20, 2023, the AI classifier is no longer available due to its low rate of accuracy. We are working to incorporate feedback and are currently researching more effective provenance techniques for text, and have made a commitment to develop and deploy mechanisms that enable users to understand if audio or visual content is AI-generated.

It makes me wonder - If OpenAI isn't good at detecting AI generated text, how reliable are other detectors?

Very well observed.

I am of the opinion that detecting AI text is extremely difficult, especially if the AI used does not have a writing pattern. And I believe that the more AI writes and evolves, the less standard it has, writing more and more in a very natural way.

So it will be more and more difficult to find out if a text is made by AI or not, just by reading it. Here on the forum, it will be the interaction with the user that will allow you to understand whether or not you are an AI.

[RickDeckard]

So it will be more and more difficult to find out if a text is made by AI or not, just by reading it. Here on the forum, it will be the interaction with the user that will allow you to understand whether or not you are an AI.

What's worse than having tools that don't accurately determine if a text is made from AI or not is the repercussion that they may have, the so called "false positive". That could easly happen in here - Imagine that a text from a user, with a shady posting behaviour, is marked as being AI generated - surely he'll face some criticism from the community (and with reason). But what if it's a "false positive"? His reputation will still be stained because AI detectors have determined that his text was "positively AI generated". Sure there are always indicators that one can take from such posts, but should we give the result of these AI detectors as proof when one of the biggest AI companies has admitted that they failed in detecting AI generated text?

[dkbit98]

It makes me wonder - If OpenAI isn't good at detecting AI generated text, how reliable are other detectors?

They are not reliable enough, maybe because AI text generators are constantly changing and it's not easy to keep track with all changes and updates.
Using several sources in combination with common sense can give better results in detection, but even them don't expect to have high precision detection.

[joker_josue]

Sure there are always indicators that one can take from such posts, but should we give the result of these AI detectors as proof when one of the biggest AI companies has admitted that they failed in detecting AI generated text?

Well, you can't. That's why I've always said that these tools aren't efficient, and certainly (unfortunately for us) they never will be. We have to find other ways - than simply copying a text and checking it in a tool, to understand what comes from an AI or not.

Honestly, I don't know how we can do this. The only idea that comes to mind is the interaction with the person. If it's an AI, sooner or later it will respond in a way that doesn't make much sense and most likely a human wouldn't respond that way.

They are not reliable enough, maybe because AI text generators are constantly changing and it's not easy to keep track with all changes and updates.
Using several sources in combination with common sense can give better results in detection, but even them don't expect to have high precision detection.

I don't think it has much to do with the fact that AI is evolving day by day. Yes, this fact is important, but it will not be the main element.

There are 1001 ways to say the same thing. Unless there is a clearly identifying pattern, it is very difficult to find out whether a text is made by an AI rather than a human. If we already know how the person habitually writes, it will be easier to detect if this way suddenly changes. Otherwise, it gets pretty difficult.

Even so, we have as an example cases of users who sold their accounts, and only several months (or years) after they were discovered. If we have a hard time figuring out the difference between the writing style of two humans. Let alone an AI that can take on multiple personalities.

[nutildah]

There are 1001 ways to say the same thing. Unless there is a clearly identifying pattern, it is very difficult to find out whether a text is made by an AI rather than a human. If we already know how the person habitually writes, it will be easier to detect if this way suddenly changes. Otherwise, it gets pretty difficult.

Even so, we have as an example cases of users who sold their accounts, and only several months (or years) after they were discovered. If we have a hard time figuring out the difference between the writing style of two humans. Let alone an AI that can take on multiple personalities.

I'm not offended by AI text that passes in my eyes for human text. I'm offended by the AI text that is obviously so, and as has been mentioned in this thread extensively, there are giveaways for such text. There are clear patterns for observing AI-written text but to see them you need to be a native English speaker and also be good at detecting patterns.

If somebody manipulates the ChatGPT / GPT-3 output to make it less detectable, I'm more-or-less fine with that as it means they have bothered to put some time into making it less mechanical and more human. Most of the time when people use AI, its because they are lazy, so they won't bother spending the time editing their posts to make them less AI.

[joker_josue]

I'm not offended by AI text that passes in my eyes for human text. I'm offended by the AI text that is obviously so, and as has been mentioned in this thread extensively, there are giveaways for such text. There are clear patterns for observing AI-written text but to see them you need to be a native English speaker and also be good at detecting patterns.

If somebody manipulates the ChatGPT / GPT-3 output to make it less detectable, I'm more-or-less fine with that as it means they have bothered to put some time into making it less mechanical and more human. Most of the time when people use AI, its because they are lazy, so they won't bother spending the time editing their posts to make them less AI.

I'm not saying there aren't some patterns. But as you say, they are not easy to detect, and have to become evident throughout several texts. But a quality AI, and with its correct use, can write things in different ways, making it difficult to create patterns. Therefore, I believe that over time it will be increasingly difficult to detect.

[dkbit98]

I don't think it has much to do with the fact that AI is evolving day by day. Yes, this fact is important, but it will not be the main element.

I wouldn't say AI is evolving at all by itself, there are people behind who are changing the code and they feed it with new information all the time.

There are 1001 ways to say the same thing. Unless there is a clearly identifying pattern, it is very difficult to find out whether a text is made by an AI rather than a human. If we already know how the person habitually writes, it will be easier to detect if this way suddenly changes. Otherwise, it gets pretty difficult.

Yeah but AI is not inventing anything new, it only uses old stuff that other people wrote before, and it combines this from several sources.
This is nothing else than mass plagiarism, and it's all coming only from bias government approved sources, so you can't get different opinion from AI.
Easy way for me to identify if something is AI or not would be talking with it for few minutes, and I can do the same thing with AI suspected members in bitcointalk forum.

[Sexylizzy2813]

How I wish the Forum would have something that can identify an AI post without stressing mods or other members who check to identify such users.
Or when posting using AI the post shouldn't be allowed, like the Forum should have an inbuilt AI detector that won't allow the user to drop the post and it should be having a red sign that the post is an AI post.

[jokers10]

How I wish the Forum would have something that can identify an AI post without stressing mods or other members who check to identify such users.
Or when posting using AI the post shouldn't be allowed, like the Forum should have an inbuilt AI detector that won't allow the user to drop the post and it should be having a red sign that the post is an AI post.

It's not so easy. Let's see on this user Adams0001. Usually he makes some edits to the texts so detectors don't easily catch him. You can probably catch him manually, but it will take lots of time. If you'll read his texts you'll see that some of them are kind of technical, people don't speak that way usually. So as human beings we can see that it is not natural. But AI detectors sometimes don't see the same.

But he made at least one big mistake:

It is difficult to determine if cryptocurrencies will survive indefinitely. Cryptocurrency is a digital or virtual asset that is secure through encryption and is decentralized, meaning it is not controlled by a single body. It is frequently used as a means of exchange and is available for trading on internet exchanges.

Cryptocurrency has grown in popularity in recent years as a means of doing transactions that are faster, cheaper, and more secure than traditional methods. However, cryptocurrency's future is uncertain, and it is possible that it will not last forever. One of the most significant difficulties for bitcoin is regulatory ambiguity. Many governments around the world are still attempting to figure out how to regulate bitcoin. This has resulted in a lack of clarity and consistency in the regulatory framework, which can lead to market uncertainty and volatility. This, in turn, may make it difficult for businesses and individuals to use cryptocurrencies, limiting its acceptance and threatening its long-term survival.

This post is easily detected by each detector with 99%+. Moreover we can check it with a plagiarism tool like text.ru and see results which are characterful for AI written: big parts taken from some other texts sometimes with whole phrases.


So now we are sure he is using AI, but how can a detector know it about some other texts with automated results of 0% probability?

I tried to get three posts of him for AI Spam Report Reference Thread, but changes he makes (sometimes he just deletes some parts of text) make it harder. Some texts have 60%+ of AI probability, some have 50%+, and I'm not motivated to spend too much time on him. So automated detectors failed.