[Experimental] Bitcointalk quality score

[Vod]

Help is welcome, thank you, Vod! What would you like to participate in?

I'll help you test your code.    When I enter my username into your tool, I want to see my score.  So build the tool lol

[*Ace*]

Help is welcome, thank you, Vod! What would you like to participate in?

I'll help you test your code.    When I enter my username into your tool, I want to see my score.  So build the tool lol

Would you like a preview of your score?

I saw something you won't like!




Honestly! You're not low quality! It's just a formula that needs to be fine-tuned.

[dkbit98]

Interesting idea, but there is no way you can measure quality in forum that easy.
I would say that with current formula your script more appropriate name is Quantity Score.
For getting more accurate Quality score you should include more factors, maybe how many members read and merited posts.

PS
OP you should release the code on Greasyfork.org website, that makes it more visible and easier to install.

[Mia Chloe]

~snip

Well I must commend you for the Idea and for writing a code for it. However personally I don't fancy the idea. This feature is barely different from the reach feature on altcoins talks and any one can bear me witness it sucks over there. Now don't feel attacked I'm not trying to back lash your work.

However you too will agree that the forum generally recognises fine posters and I personally believe that it's enough. Barely going through a user's profile you can tell if they are a shit poster or not.

[Upgrade00]

Anyone who looks at a profile that has over 5000 merits their first instinct will be that this is an elite user regardless of the amount of posts they have made. A system that will function as well or better than this eye test will need to take a lot of factors into account including age of account, number of posts made and more.

You have a good idea with the concept of the idea but in practice I don't think it's a fair judge. The forum is very community based, this means lots of discussions would be made purely with the intention to communicate with other users and not to impact quality. This is what happens when people post naturally and not with the intention to chase merits, but they will offer quality at times it becomes necessary.

[Ambatman]

This is great and though still biased towards merit even if it is the standard metric.
I have some questions though

1. Does it account for deleted posts /thread and also does it account for merit gotten from a deleted thread?

2. I have seen quality posters that frequent the off-topic board that post without a care of merit, does that make them low quality?

3. Economics board don't really get that much attention like some other boards and there are quality posters with to the metric you using that would brand them low.


I know no system is perfect but I just feel bad for ChartBuddy

[*Ace*]

Interesting idea, but there is no way you can measure quality in forum that easy.
I would say that with current formula your script more appropriate name is Quantity Score.
For getting more accurate Quality score you should include more factors, maybe how many members read and merited posts.

PS
OP you should release the code on Greasyfork.org website, that makes it more visible and easier to install.

Your observation is correct but difficult to implement! As already mentioned, the tool does not check the content of posts, nor does it check where the posts are written.
Basically, it fetches the user profile page and obtains the data it needs to examine. It also takes the time data from bitlist to see if the user has recent activity or is inactive.
What it returns is a snapshot of what the community thinks of me or you, or of fillippone or LoyceV, babo, etc.
It does not examine quality in the strict sense of the word, but only what each of us has done during our time on this forum.
We wouldn't even need a tool to understand that, on average, I have received 0.8 Merit for each of my posts! But the formula would serve to eliminate false positives as much as possible, not reward shitposters, and understand, for better or worse, how we have behaved on the forum, whether we have been appreciated or not.

As far as you are concerned, I can openly say that vod has created a truly fantastic and impressive tool. It will not replace the Merit system, but rather serve as a different form of "voting" to understand how much we appreciate the contributions of others, without limiting the vote. I don't know if I'm explaining myself well, but in short, to give 1 Merit, I first have to earn 2, which in itself is a limitation.
I'm not saying that the Merit system has problems, but simply that it's not entirely fair!
In my opinion, this is precisely what gives rise to merit abusers.

I almost forgot! The tool will be released on Greasyfork as soon as I have a stable formula, so it will be clearly visible to everyone.

[sleepfirefly]

First, I commend your effort in putting this together, kudos to you!

But I tell you, post, and only post can be the true decider of whether a user is a quality poster. Once merits are considered, you missed it.

it is the simplistic way of determining if one is at least contributing something others find worth sending merits to though it is true that it’s not always the most accurate metric of quality but the tool can also show how active of a person has been recently because no matter how many the merits but they have not been receiving one in a long time, there must be a reason why

[Vod]

Honestly! You're not low quality! It's just a formula that needs to be fine-tuned.

This is what I mean by helping you test.  I'm not the type of guy to add you to my distrust list if you say something mean.  

Anyone who looks at a profile that has over 5000 merits their first instinct will be that this is an elite user regardless of the amount of posts they have made.

Not entirely true - posts often get extra merits because the source has too many.  Number of merits is not an indication of merit, though it should be. 

[TypoTonic]

Interesting idea, but there is no way you can measure quality in forum that easy.
I would say that with current formula your script more appropriate name is Quantity Score.
For getting more accurate Quality score you should include more factors, maybe how many members read and merited posts.
-snip-

Your observation is correct but difficult to implement! As already mentioned, the tool does not check the content of posts, nor does it check where the posts are written.
Basically, it fetches the user profile page and obtains the data it needs to examine. It also takes the time data from bitlist to see if the user has recent activity or is inactive.
-snip-

If only user profiles are checked, then maybe you can include the data from BPIP? I think it's worth considering these as additional factors:

Code:

Has earned 'x' merit 'y' times, from 'z' profiles

-snip- I don't know if I'm explaining myself well, but in short, to give 1 Merit, I first have to earn 2, which in itself is a limitation.
I'm not saying that the Merit system has problems, but simply that it's not entirely fair!
In my opinion, this is precisely what gives rise to merit abusers.

A perfect system doesn't exist, there will always be flaws. I honestly think (earned) merits would have more value if there was a sending limit of 1 per post, from when it was first introduced. (But that's just my opinion of course, sorry if it's a bit off-topic)

[FinneysTrueVision]

The tool might not be perfect in its current form, but it still has potential. I could see an automated scoring system being useful for determining who the worst posters are in a campaign. Too often I see users who are legendary rank but they hardly earn any merit. It is easier for those users to fly under the radar if they are in large campaigns with lots of participants, where managers might not have the time to go to every individual profile to see how frequently they earn merit.

[fillippone]

For getting more accurate Quality score you should include more factors, maybe how many members read and merited posts.

I think, though difficult to implement according to *Ace*, the number of people who merited a post is always significant.
I have the suspect this was also a part of the recipe used in the last ddmrddmr merit shower.
Anyway, as always said, the true value of a post cannot be deduced by a formula, because only a human reader can assess it, and in the very moment you determine the formula, you immediately also fix the way to game that formula.

[Upgrade00]

Not entirely true - posts often get extra merits because the source has too many.  Number of merits is not an indication of merit, though it should be. 

Except there is some sort of bias, this situation will not be happening with the same user over and over again to boost their score. You'll have to be doing something right to constantly get merits, that's why I set the bar as high as 5000 or we could take it higher to 8k or 10k, at that point it's >1% of the community, and I think that should count as elite. If we raise the bar higher than that, we are splitting hairs

[LoyceV]

After thinking about it a bit more, I think inactivity should reduce the quality score more than just based on posts in the last 120 days. Did satoshi do good things? Yes! Is he a quality poster? Well..... He's not even "a poster"! Users like nullius or n0nce made good quality posts, but they were just a blip on Bitcointalk's timeline.
It's going to be really difficult to put all possible scenarios in one formula.

[Mahiyammahi]

Well your Idea Itself has a great potential. But to make this reality you need to utilize more and develop it further. So if it's about Quality Vs Quantity here's an Idea from me.

Why not exclude Games & Round Section , Merit giveaway threads , Threads that includes off topic disscusssion, Gambling section From your countdown if it's possible. These are not adding value to a user's profile really. We hardly see any  merit transaction on Gambling section and most user's go there just to fill their quota. So value are added when a user's getting merit from other section.

If it's possible filter them out and make a script it would give us a better idea. If a user do not have any skill to get merits without merit giveaways thread or where merit transaction are high then Imo they don't deserve to get a qualifications score.

One more thing
Reputation Can't be replaced by some algorithm, no matter how hard you try.

[*Ace*]

Okay, I've made a small change to the formula.
Now, new members' exploits shouldn't have a huge impact.
So if a user has 10 posts and lots of Merits, they can't have the same score as a user with 1,000 posts and lots of Merits.
The formula adjusts according to the age of the profile.

Code:

Q_hist = M_tot / max(P_tot, 1)

Q_120 = (P_120 > 0) ? (M_120 / P_120) : 0

Age_factor = min( log(Age_days + 1) / log(3650), 1 )

Activity_factor = min( P_120 / 50, 1 )

Score =
    0.5 * Q_hist * Age_factor
  + 0.3 * Q_120  * Activity_factor
  + 0.2 * Q_hist

Quality_score =
    (P_120 > 0)
    ? (0.6 * Q_hist + 0.4 * Q_120)
    : Q_hist

Code:

P_tot      : Total number of posts of the user
M_tot      : Total number of Merit points received by the user
P_120      : Number of posts in the last 120 days
M_120      : Number of Merit points received in the last 120 days
Age_days   : Age of the user profile in days (current date - registration date)

Q_hist     : Historical quality ratio (M_tot / max(P_tot, 1))
Q_120      : Recent 120-day quality ratio (M_120 / P_120 if P_120 > 0, otherwise 0)
Age_factor : Factor representing profile age, capped at 1
Activity_factor : Factor representing recent activity, capped at 1

Score          : Weighted score combining historical quality, recent quality, and age/activity factors
Quality_score  : Final quality score used for ranking, using a mix of Q_hist and Q_120

These are some results that are a bit more consistent. What do you think?

[winspiral]

Sorry I have understood nothing...
could someone explain ...

[*Ace*]

After thinking about it a bit more, I think inactivity should reduce the quality score more than just based on posts in the last 120 days. Did satoshi do good things? Yes! Is he a quality poster? Well..... He's not even "a poster"! Users like nullius or n0nce made good quality posts, but they were just a blip on Bitcointalk's timeline.
It's going to be really difficult to put all possible scenarios in one formula.

I could impose a penalty based on the length of absence from the forum.
Example: inactivity > 365 days reduces score by 10%.

Well your Idea Itself has a great potential. But to make this reality you need to utilize more and develop it further. So if it's about Quality Vs Quantity here's an Idea from me.

Why not exclude Games & Round Section , Merit giveaway threads , Threads that includes off topic disscusssion, Gambling section From your countdown if it's possible. These are not adding value to a user's profile really. We hardly see any  merit transaction on Gambling section and most user's go there just to fill their quota. So value are added when a user's getting merit from other section.

If it's possible filter them out and make a script it would give us a better idea. If a user do not have any skill to get merits without merit giveaways thread or where merit transaction are high then Imo they don't deserve to get a qualifications score.

One more thing
Reputation Can't be replaced by some algorithm, no matter how hard you try.

Your considerations are more than legitimate. The problem is that with the data available, I cannot filter it as you requested. It would require a database from which to draw the data and filter it, a database that unfortunately I do not have and do not know if it is available.

[babo]

Honestly! You're not low quality! It's just a formula that needs to be fine-tuned.

exactly, and when you're done, as I proposed to Vod for his very nice project I can audit the code on github
I thought with him, but it goes for you too Ace
I can check and make a post that you can link in the OP of the thread and say.. babooooooo¹ validated this code



[1] for @Vod

[*Ace*]

Honestly! You're not low quality! It's just a formula that needs to be fine-tuned.

exactly, and when you're done, as I proposed to Vod for his very nice project I can audit the code on github
I thought with him, but it goes for you too Ace
I can check and make a post that you can link in the OP of the thread and say.. babooooooo¹ validated this code



[1] for @Vod

While you were writing this post, I updated OP and released the code on Greasyfork, so if anyone wants to see it or install it, they can do so easily.

Thanks in advance, babo.