A wave of bans: 400 yesterday, 300 the day before. What changed?

[Quickseller]

Does anyone has any real information about what tool is used and by whom to actually make these bans happen?

Yes I'm curious

Are you asking who is reporting the plagiarism to the moderators? Or do you wish to know which moderator is actually banning forum members?

There appear to be multiple people reporting plagiarism, although suchmoon is very clearly one person running a bot to check for plagiarism and I would not trust her to not use her bot to check for plagiarism against her enemies (very broadly speaking).

I presume that Hilariousandco is at least involved in reviewing the reports and banning people. Many who have been banned are addressing their appeal threads to him, so I would reasonably conclude that he is sending a PM when a person is banned. I don’t think he is a well known enough of a moderator for this many people to address their appeal threads to him and so few/none to other moderators. He also has been able to produce the specific posts that have been reported when people are asking about their ban. I would be surprised if he is the only one working on this though.

[1713254458]

1713254458

[1713254458]

1713254458

[1713254458]

1713254458

[LoyceV]

He also has been able to produce the specific posts that have been reported when people are asking about their ban.

I assume hilariousandco can just view all reports, which makes it easy to find the specific post.

[Quickseller]

He also has been able to produce the specific posts that have been reported when people are asking about their ban.

I assume hilariousandco can just view all reports, which makes it easy to find the specific post.

This is true. He was also the first to actually care about the problem years before any serious push was done to address it and find instances of plagiarism in masse. He also appears to care about addressing the problem a lot. I would be seriously surprised to learn that he is in no way involved.

[akamit]

There appear to be multiple people reporting plagiarism, although suchmoon is very clearly one person running a bot to check for plagiarism and I would not trust her to not use her bot to check for plagiarism against her enemies (very broadly speaking).

I had assumed that the bot will go through 1 to 2601082 accounts and will check every post to find plagiarism.

[Vod]

There appear to be multiple people reporting plagiarism, although suchmoon is very clearly one person running a bot to check for plagiarism and I would not trust her to not use her bot to check for plagiarism against her enemies (very broadly speaking).

I had assumed that the bot will go through 1 to 2601082 accounts and will check every post to find plagiarism.

That's half a billion posts.   

[Quickseller]

There appear to be multiple people reporting plagiarism, although suchmoon is very clearly one person running a bot to check for plagiarism and I would not trust her to not use her bot to check for plagiarism against her enemies (very broadly speaking).

I had assumed that the bot will go through 1 to 2601082 accounts and will check every post to find plagiarism.

If Vod is correct about the number of posts, that would mean taking over 5,000 days assuming one post was checked per second.

I don’t think this is what’s happening.

Also, without saying as many words, suchmoon strongly implied she would not report lauda plagiarism if it was found.

[suchmoon]

That's half a billion posts.  

Only 50 million

Still a quadrillion pairs just to crosscheck post-to-post copies inside the forum so good luck to anyone attempting that.

[hilariousandco]

He also has been able to produce the specific posts that have been reported when people are asking about their ban.

I assume hilariousandco can just view all reports, which makes it easy to find the specific post.

Bingo. I don't message anyone, but I'm also  - arguably - the most active moderator, or at least the one who is active in posting publicly, so people tend to go to me rather than the admins or anyone else.

[akamit]

If Vod is correct about the number of posts, that would mean taking about 5,000 days assuming one post was checked per second.

5000 days is wrong... It will take 593 days to scan the current total post count.

Total forum posts 51,269,556 / 86400 secs (24hrs) = 593.40 days

I hope the calculation is correct.

I don’t think this is what’s happening.

Also, without saying as many words, suchmoon strongly implied she would not report lauda plagiarism if it was found.

No comment here... as I don't know.

[asche]

I am mostly interested in the tool - the bot - used to actually report the plagiarists.

The who is mostly irrelevant, unless the who talks about said tool he/she/it uses.

[Quickseller]

He also has been able to produce the specific posts that have been reported when people are asking about their ban.

I assume hilariousandco can just view all reports, which makes it easy to find the specific post.

Bingo. I don't message anyone, but I'm also  - arguably - the most active moderator, or at least the one who is active in posting publicly, so people tend to go to me rather than the admins or anyone else.

Fair enough. Sounds like a reasonable explanation.

If Vod is correct about the number of posts, that would mean taking about 5,000 days assuming one post was checked per second.

5000 days is wrong... It will take 593 days to scan the current total post count.

Total forum posts 51,269,556 / 86400 secs (24hrs) = 593.40 days

The 5,000 day figure was based on Vods half billion post figure.

I am mostly interested in the tool - the bot - used to actually report the plagiarists.

Most likely, the bot scrapes posts on the forum, compiles them into a database along with who posted and the date/time of the post, and cross references the posts among other entries in the database. Presumably other sources of writings are also compiled into the database, such as news articles, blog posts and similar, and the scraped posts are also crossed checked against these entires.

It would be very similar to how google or any other search engine works, except the database is run on your own server. 

[LoyceV]

I had assumed that the bot will go through 1 to 2601082 accounts and will check every post to find plagiarism.

That's half a billion posts.   

Your msgID was 51311881 (51,311,881). That's "only" 51 million posts

If Vod is correct about the number of posts, that would mean taking over 5,000 days assuming one post was checked per second.

You can download one page (up to 20 posts) per second. In ideal circumstances this would take just a month, in reality probably 2 months.

Still a quadrillion pairs just to crosscheck post-to-post copies inside the forum so good luck to anyone attempting that.

I've looked at it before, finding duplicate posts wasn't that hard. The problem was the very large number of BS-posts like "good project". It's not plagiarism, it's useless, but there are far too many similar posts to manually report them all.

I think a combination of checksum with Linux commands "sort" and "uniq" can be used to find exact duplicates pretty fast. That solves the problem of a quadrillion checks as it'll use a well tested and very fast sorting algorithm. If someone can get me 51 million posts (excluding quotes and probably smileys), I'll give it a try.

[Quickseller]

If Vod is correct about the number of posts, that would mean taking over 5,000 days assuming one post was checked per second.

You can download one page (up to 20 posts) per second. In ideal circumstances this would take just a month, in reality probably 2 months.

Ahh yes, that is my mistake. I am not sure what the distribution of page lengths look like, but some pages are shorter than 20 posts, for example for threads with 21 posts, the second page will have one post, and a thread with 3 posts will have one page with 3 posts. So long as the average thread length is above 30 posts, it would take an upper bound of 2 months to scrap all the posts.

he problem was the very large number of BS-posts like "good project".

You should be able to filter these types of posts out. Ditto with posts under a certain length that would not be reasonably plagiarism.

[qwk]

I am mostly interested in the tool - the bot - used to actually report the plagiarists.

The who is mostly irrelevant, unless the who talks about said tool he/she/it uses.

I'm not sure if my "prime suspect" has been mentioned yet, but feel free to review an older thread on the topic:
https://bitcointalk.org/index.php?topic=5032322

[Quickseller]

I am mostly interested in the tool - the bot - used to actually report the plagiarists.

The who is mostly irrelevant, unless the who talks about said tool he/she/it uses.

I'm not sure if my "prime suspect" has been mentioned yet, but feel free to review an older thread on the topic:
https://bitcointalk.org/index.php?topic=5032322

I am not sure who your "prime suspect" is, but I found a "confession" in that thread:

I'm experimenting with some NLP techniques for plagiarism detection and the results are promising although scalability is a bit of an issue. Currently working just on comparing Bitcointalk posts (not to outside sources).

I experimented with n-grams a little bit and couldn't find a good value. Low n yields too many false positives, high n doesn't detect spinners, etc. So I'm using a mixture of algorithms and base the decision on the pattern of the results of those algorithms - e.g. if the similarity of two texts using algorithm A is 70%, then union/intersect/otherwise manipulate the texts, run algorithm B, if it scores 90% then run algorithm C to eliminate false positives - made up numbers but you get the idea. Works ok-ish, but as I mentioned it doesn't scale well and I need to do more testing on larger samples.

[HCP]

The only way to satisfy your curiosity is to ask someone who has that information via PM.

Given the Meta/Reputation drama revolving around some PM's being made public in recent times... You might find a few people quite reluctant to put anything in a PM

[Quickseller]

"confession"

Brilliant detective work, Quicksy. May I suggest investigating bears and popes next.

I literally have "snitch" under my avatar, speaking of which - I was wearing Stitch for months. I actually explained to you personally how I look for plagiarism when you decided to lie about it, which happens a lot for some reason. Early onset Alzheimer's or just trying to save your account farm by attacking people who're cleaning up the forum from garbage like you?

The bot is likely a myth though. I doubt there is anyone running an actual bot (autonomous software) to find and report posts. I report a shitload of posts - on a good month probably in double digits as a percentage of all reports - and I still manually review and report every single post. Just using some software tools more tailored to the task than a simple Google search.

[...]

I found it weird that you were dancing around the subject of you actually running a bot, expect for the sarcastic comment at the end of your post.....until I remembered this:

I'm involved in what could be considered a service to the community and there are some expenses (details available to serious bidders on request). I've been footing those bills on my own for the last ~year but my SO got a hold of the last one... long story short, that's no longer a viable option

Neither of the posts explicitly say if you are presently running a bot, and the post in which you sell your avatar doesn't explicitly say what the money will be used for, leaving open the possibility that either you could be using the money to finance the cost of the bot, and to reimburse your family the previous cost of the bot. No big deal, there isn't any reason to be forced to disclose what money that is legitimately obtained is going to be used for.

No one cares if the person reporting the posts actually looks at the reports before submitting them, the end result is still the same. Frankly, so long as any bot used uses sufficient logic to weed out common phrases that would not be plagiarism, there shouldn't be any concerns with automatically reporting posts. Although based on who I believe is most likely to have created the bot, I have my doubts the person's junk science would allow him to sufficiently weed out these false positives.

Back on track, I think it is pretty clear you are still using the bot, and I think you are expending your own money to pay for these expenses. I also think you are probably not explicitly lying, but are making statements that you believe will lead your SO to believe you are *not* any longer paying these expenses yourself (and that he actually believes this). In some regards, this kind of deception to a SO, or someone you have a close personal relationship with is worse than explicitly lying because it leaves open the possibility to deny being deceptive when caught and confronted (and asked if you were previously lying).

I am not sure if you paid, or are paying, or were but no longer are paying the person who I believe to have created the bot. I believe him to be a scammer, troll, and a spammer, and not one to care about others enough to want to make this kind of tool for free. On the other hand, he has a history of smearing anyone who calls out his shady behavior, and holds a grudge, so I can't rule this person out from being willing to create a tool to use his junk science to further grind on his ax so to speak, that eventually got repurposed and/or modified to catch people plagiarizing.

Knowing my observations of your behavior on the forum, and who I know you align with, I am not surprised to see deception on your part. 

[deisik]

Feel free to ask theymos what the rate of false positives is in my reports, lying asshat.

And what is that rate (just curious)?

To make things clear, I don't have any grudges against you and I don't even know you as we don't seem to have been engaged in a lot of converse here. I just want to know what the "tool" will report if someone posts a lengthy quote from a prominent figure (that doesn't require a reference for being well known to every literate person) and then someone else repeats it in a different context or even in a different thread. Will this get reported as plagiarism?

[bones261]

Feel free to ask theymos what the rate of false positives is in my reports, lying asshat.

And what is that rate (just curious)?

To make things clear, I don't have any grudges against you and I don't even know you as we don't seem to have been engaged in a lot of converse here. I just want to know what the "tool" will report if someone posts a lengthy quote from a prominent figure (that doesn't require a reference for being well known to every literate person) and then someone else repeats it in a different context or even in a different thread. Will this get reported as plagiarism?

From what I can tell, the tool will only report when a human being interacts with it. So it all depends on what the tool user thinks. Then a mod has to review the report an make a determination as well. So, basically two human beings have to agree that the post is plagiarism before a ban occurs. I suspect that if someone really had a bot that totally automated the whole process, the staff would probably be overwhelmed with a huge back load.

[suchmoon]

Feel free to ask theymos what the rate of false positives is in my reports, lying asshat.

And what is that rate (just curious)?

Low enough for no one to be concerned about it, except Quickseller.

To make things clear, I don't have any grudges against you and I don't even know you as we don't seem to have been engaged in a lot of converse here. I just want to know what the "tool" will report if someone posts a lengthy quote from a prominent figure (that doesn't require a reference for being well known to every literate person) and then someone else repeats it in a different context or even in a different thread. Will this get reported as plagiarism?

I won't report a well-known quote if it's missing a reference, if that's what you're asking. The "tool" will often flag such things because it can't predict the sheer variety of shitty quoting abundant on this forum. Moderators are generally quite literate people too, so they will mark reports as "Bad" if I'm not familiar with a particular genre of folklore.